Log management with tiered storage

Which brands AI mentions

2. Datadog Log Management

Offers log archives in customer-owned cloud storage and Archive Search, which queries historical logs directly without requiring advance rehydration.

• Storage: Customer-owned object storage, including supported S3 and Google Cloud storage tiers.
• Search: On-demand archive queries; results stream back as the archive is scanned.
• Best for: Teams wanting managed SaaS logging and occasional audit searches.
• Watch out for: Scan charges and additional indexing costs if results need to remain available longer or be rehydrated.

Documentation: Archive Search and Log archives.docs.datadoghq.com+1

3. Amazon OpenSearch Service

Provides lower-cost storage options such as UltraWarm, plus searchable snapshots for supported configurations. This can reduce the cost of keeping historical data available to search.

• Storage: S3-backed storage options.
• Search: Depends on the selected storage tier and snapshot architecture.
• Best for: AWS-centric environments that want integrated log analytics.
• Watch out for: Confirm which tier supports your required query workflow; not every archival option offers identical search behavior.

Documentation: UltraWarm storage.

4. Sumo Logic

Its Continuous, Frequent, and Infrequent data tiers let organizations match log retention and search frequency to the workload. The Infrequent tier is intended for data that is queried less often and uses a pay-per-search model.

• Storage: Vendor-managed data tiers rather than simply moving data to your own object-storage bucket.
• Search: Supported across tiers, with different query and analytics capabilities.
• Best for: Compliance and operational logs that must remain queryable but are accessed infrequently.
• Watch out for: Tier availability and dashboard, scheduled-search, and query limitations vary by plan.

Documentation: Data tiers.www.sumologic.com+2

5. Grafana Loki

Stores log chunks in inexpensive object storage and uses a relatively compact index based on labels. This can make long-term retention economical, especially for high-volume infrastructure logs.

• Storage: S3-compatible object storage and supported cloud object stores.
• Search: Query historical chunks through LogQL, subject to retention, availability, and query limits.
• Best for: Kubernetes, cloud-native infrastructure, and engineering teams comfortable operating Grafana's logging stack.
• Watch out for: Search performance depends on label design, query scope, and the amount of historical data scanned.

Documentation: Loki storage configuration.

6. Splunk

SmartStore separates indexer compute from remote storage, using remote object storage and a local cache. It is useful for managing storage costs while preserving a searchable data workflow, though remote retrieval and cache behavior matter.

• Storage: Remote object storage, including supported S3-based configurations.
• Search: Searchable data managed through SmartStore; this is not the same as every type of offline archive being immediately searchable.
• Best for: Organizations already invested in Splunk Enterprise or Splunk Cloud.
• Watch out for: Validate the exact archive design and retrieval latency, particularly if using a separate non-searchable archive.

Background: Splunk Cloud retention and archiving.lantern.splunk.com+2

Quick comparison

Sources: vendor documentation for Datadog, Sumo Logic, Amazon OpenSearch Service, and Splunk.docs.datadoghq.com+2

My recommendation

• For direct searches across older data: Start with Elastic or Datadog Archive Search.
• For AWS-native infrastructure: Evaluate Amazon OpenSearch Service.
• For predictable, lower-cost retention with occasional queries: Evaluate Sumo Logic's Infrequent tier.
• For very high log volumes and lower indexing overhead: Evaluate Grafana Loki.

Three audit requirements to verify before buying

1. Searchability: Can you search several years of logs immediately, or must you restore or re-index them?
2. Evidence integrity: Does the design support immutable retention, restricted deletion, access logging, and the retention periods required by your policies?
3. Total cost: Compare storage, ingestion, indexing, query scans, retrieval, and data export—not just the advertised storage price.

One important caveat: searchable archival storage does not automatically guarantee compliance. You need to configure retention controls and evidence-handling procedures for your specific audit requirements.

One question that would help narrow the options: Are you primarily looking for a managed cloud service, or can you run the logging platform yourself? That makes a substantial difference to both cost and architecture.