BigQuery log storage

This view allows users of the BigQuery log storage method to view the storage status of Enriched Logs ingested by , which are maintained in Hot and Cold storage.

  • Storage Timeline: Dark blue indicates hot storage, and light blue indicates cold storage. Beneath the Timeline the total size of stored data is displayed.

  • Hot Storage: Enriched Logs are stored and immediately available to search and review for at least seven days.

  • Cold Storage: Enriched Logs are not available for search or review without first bringing the log data for the data in question into Hot Storage. To do this, click the + Restoration Task button.

  • Performing a Log Restoration Task

Clicking the + Restoration Task button displays the New Restoration Task window:

To initiate restoration of cold storage logs:

  1. Enter a name for the task in the RESTORATION TASK NAME field (Note: The name cannot contain special characters).
  2. Use the FROM field and the HOUR field below it to specify the starting point of the time window of cold logs that will be restored.
  3. Use the TO field and the HOUR field below it to specify the ending point of the time window of cold logs that will be restored (Note: The maximum time window allowed for a single restoration task is 7 days).
  4. Click the Restore button - the task will appear on the screen under Restoration Tasks.

Restoration Tasks

This section of the BigQuery log storage management screen provides information about log storage restoration tasks that are currently active:

  • RESTORATION NAME - The name given to the currently active log storage restoration task by its creator.

  • RESTORATION DATES- The time window of the restored cold logs that the restoration task has made available for review.

  • CREATED BY - The person who is conducting the currently active log storage restoration task.

  • DATE CREATED - The date that the currently active log storage restoration task was initiated.

  • RESTORED - The percentage of targeted logs that were successfully restored. Clicking the refresh button to the right updates the progress indicator. Other status indicators include Partial, Unknown, Error, and Failed. When restoration is only partial, the percentage of restoration is shown in yellow with a help icon that details how many jobs were completed when clicked. Partial and Failed statuses provide the option to re-attempt the restoration as the last column in the table via the refresh icon.

  • EXPIRES IN - The date when the restored logs will cease being available for review and be returned to cold storage.

Restoration History

This section of the BigQuery log storage management screen provides information about log storage restoration tasks that are currently active:

  • RESTORATION NAME - The name of the historical log storage restoration task given by its creator.

  • RESTORATION DATES - The time window of the restored cold logs that the restoration task made available for review.

  • CREATED BY - The person who conducted the historical log storage restoration task.

  • DATE CREATED - The date that the historical log storage restoration task was initiated.

  • RESTORED - The percentage of targeted logs that were successfully restored.

  • EXPIRED - The date when the restored logs ceased being available for review and were returned to cold storage.