Log Analytics Workspace Architecture: Tables, Tiers, and Access Design

The cost model rewards sorting telemetry by how it will be read. Analytics tables carry the interactive tier, full KQL, alerts, dashboards, priced accordingly, and reserved for what humans and alert rules actually query. Basic tables take the high volume operational logs queried occasionally and simply, at a fraction of the ingestion price with limited KQL and no alerting. Auxiliary tables take the compliance and forensic flows, cheapest ingestion, queried rarely and slowly, the tier the Sentinel post flagged for the keep everything mandates. Sort honestly, verbose container stdout is rarely Analytics grade, and enforce the sorting upstream with data collection rule transformations: KQL executed at ingestion that drops the debug noise, trims the fat columns, redacts what should never land, and routes rows to the right tables, the single highest leverage cost control in the observability stack because it taxes data at the door instead of storing regret. Commitment tiers then price the survivors, with the FinOps rhythm watching ingestion by table monthly, the top ten tables report being where every workspace review starts and usually where it finds the accidental gigabytes.

Retention, Archive, and the Long Memory

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

Table Plans and the Door Tax

The cost model rewards sorting telemetry by how it will be read. Analytics tables carry the interactive tier, full KQL, alerts, dashboards, priced accordingly, and reserved for what humans and alert rules actually query. Basic tables take the high volume operational logs queried occasionally and simply, at a fraction of the ingestion price with limited KQL and no alerting. Auxiliary tables take the compliance and forensic flows, cheapest ingestion, queried rarely and slowly, the tier the Sentinel post flagged for the keep everything mandates. Sort honestly, verbose container stdout is rarely Analytics grade, and enforce the sorting upstream with data collection rule transformations: KQL executed at ingestion that drops the debug noise, trims the fat columns, redacts what should never land, and routes rows to the right tables, the single highest leverage cost control in the observability stack because it taxes data at the door instead of storing regret. Commitment tiers then price the survivors, with the FinOps rhythm watching ingestion by table monthly, the top ten tables report being where every workspace review starts and usually where it finds the accidental gigabytes.

Retention, Archive, and the Long Memory

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

The default is one central workspace per region of significance, because correlation is the product, the App Insights trace joining the AKS logs joining the firewall logs works only when they share a workspace, and RBAC happens at finer grains than workspace boundaries anyway. The legitimate exceptions, each argued in earlier posts: the dedicated Sentinel workspace where security’s retention and access diverge hard, regional workspaces where data residency laws pin telemetry, and the rare tenant boundary case for service providers. What the topology should never encode is org chart vanity, a workspace per team recreates the silo problem observability exists to solve, and cross workspace queries are a tax, not a pattern. Access design inside the workspace does the real segregation: resource context access means a team granted read on their resources can query those resources’ logs and no one else’s through the resource’s Logs blade, table level RBAC restricts the sensitive tables (the sign in logs, anything with PII surviving the redaction posts) to the teams with cause, and the granular roles keep the reader and contributor lines clean.

Table Plans and the Door Tax

The cost model rewards sorting telemetry by how it will be read. Analytics tables carry the interactive tier, full KQL, alerts, dashboards, priced accordingly, and reserved for what humans and alert rules actually query. Basic tables take the high volume operational logs queried occasionally and simply, at a fraction of the ingestion price with limited KQL and no alerting. Auxiliary tables take the compliance and forensic flows, cheapest ingestion, queried rarely and slowly, the tier the Sentinel post flagged for the keep everything mandates. Sort honestly, verbose container stdout is rarely Analytics grade, and enforce the sorting upstream with data collection rule transformations: KQL executed at ingestion that drops the debug noise, trims the fat columns, redacts what should never land, and routes rows to the right tables, the single highest leverage cost control in the observability stack because it taxes data at the door instead of storing regret. Commitment tiers then price the survivors, with the FinOps rhythm watching ingestion by table monthly, the top ten tables report being where every workspace review starts and usually where it finds the accidental gigabytes.

Retention, Archive, and the Long Memory

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

The cost model rewards sorting telemetry by how it will be read. Analytics tables carry the interactive tier, full KQL, alerts, dashboards, priced accordingly, and reserved for what humans and alert rules actually query. Basic tables take the high volume operational logs queried occasionally and simply, at a fraction of the ingestion price with limited KQL and no alerting. Auxiliary tables take the compliance and forensic flows, cheapest ingestion, queried rarely and slowly, the tier the Sentinel post flagged for the keep everything mandates. Sort honestly, verbose container stdout is rarely Analytics grade, and enforce the sorting upstream with data collection rule transformations: KQL executed at ingestion that drops the debug noise, trims the fat columns, redacts what should never land, and routes rows to the right tables, the single highest leverage cost control in the observability stack because it taxes data at the door instead of storing regret. Commitment tiers then price the survivors, with the FinOps rhythm watching ingestion by table monthly, the top ten tables report being where every workspace review starts and usually where it finds the accidental gigabytes.

Retention, Archive, and the Long Memory

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

The default is one central workspace per region of significance, because correlation is the product, the App Insights trace joining the AKS logs joining the firewall logs works only when they share a workspace, and RBAC happens at finer grains than workspace boundaries anyway. The legitimate exceptions, each argued in earlier posts: the dedicated Sentinel workspace where security’s retention and access diverge hard, regional workspaces where data residency laws pin telemetry, and the rare tenant boundary case for service providers. What the topology should never encode is org chart vanity, a workspace per team recreates the silo problem observability exists to solve, and cross workspace queries are a tax, not a pattern. Access design inside the workspace does the real segregation: resource context access means a team granted read on their resources can query those resources’ logs and no one else’s through the resource’s Logs blade, table level RBAC restricts the sensitive tables (the sign in logs, anything with PII surviving the redaction posts) to the teams with cause, and the granular roles keep the reader and contributor lines clean.

Table Plans and the Door Tax

The cost model rewards sorting telemetry by how it will be read. Analytics tables carry the interactive tier, full KQL, alerts, dashboards, priced accordingly, and reserved for what humans and alert rules actually query. Basic tables take the high volume operational logs queried occasionally and simply, at a fraction of the ingestion price with limited KQL and no alerting. Auxiliary tables take the compliance and forensic flows, cheapest ingestion, queried rarely and slowly, the tier the Sentinel post flagged for the keep everything mandates. Sort honestly, verbose container stdout is rarely Analytics grade, and enforce the sorting upstream with data collection rule transformations: KQL executed at ingestion that drops the debug noise, trims the fat columns, redacts what should never land, and routes rows to the right tables, the single highest leverage cost control in the observability stack because it taxes data at the door instead of storing regret. Commitment tiers then price the survivors, with the FinOps rhythm watching ingestion by table monthly, the top ten tables report being where every workspace review starts and usually where it finds the accidental gigabytes.

Retention, Archive, and the Long Memory

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

Table Plans and the Door Tax

The cost model rewards sorting telemetry by how it will be read. Analytics tables carry the interactive tier, full KQL, alerts, dashboards, priced accordingly, and reserved for what humans and alert rules actually query. Basic tables take the high volume operational logs queried occasionally and simply, at a fraction of the ingestion price with limited KQL and no alerting. Auxiliary tables take the compliance and forensic flows, cheapest ingestion, queried rarely and slowly, the tier the Sentinel post flagged for the keep everything mandates. Sort honestly, verbose container stdout is rarely Analytics grade, and enforce the sorting upstream with data collection rule transformations: KQL executed at ingestion that drops the debug noise, trims the fat columns, redacts what should never land, and routes rows to the right tables, the single highest leverage cost control in the observability stack because it taxes data at the door instead of storing regret. Commitment tiers then price the survivors, with the FinOps rhythm watching ingestion by table monthly, the top ten tables report being where every workspace review starts and usually where it finds the accidental gigabytes.

Retention, Archive, and the Long Memory

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

The default is one central workspace per region of significance, because correlation is the product, the App Insights trace joining the AKS logs joining the firewall logs works only when they share a workspace, and RBAC happens at finer grains than workspace boundaries anyway. The legitimate exceptions, each argued in earlier posts: the dedicated Sentinel workspace where security’s retention and access diverge hard, regional workspaces where data residency laws pin telemetry, and the rare tenant boundary case for service providers. What the topology should never encode is org chart vanity, a workspace per team recreates the silo problem observability exists to solve, and cross workspace queries are a tax, not a pattern. Access design inside the workspace does the real segregation: resource context access means a team granted read on their resources can query those resources’ logs and no one else’s through the resource’s Logs blade, table level RBAC restricts the sensitive tables (the sign in logs, anything with PII surviving the redaction posts) to the teams with cause, and the granular roles keep the reader and contributor lines clean.

Table Plans and the Door Tax

The cost model rewards sorting telemetry by how it will be read. Analytics tables carry the interactive tier, full KQL, alerts, dashboards, priced accordingly, and reserved for what humans and alert rules actually query. Basic tables take the high volume operational logs queried occasionally and simply, at a fraction of the ingestion price with limited KQL and no alerting. Auxiliary tables take the compliance and forensic flows, cheapest ingestion, queried rarely and slowly, the tier the Sentinel post flagged for the keep everything mandates. Sort honestly, verbose container stdout is rarely Analytics grade, and enforce the sorting upstream with data collection rule transformations: KQL executed at ingestion that drops the debug noise, trims the fat columns, redacts what should never land, and routes rows to the right tables, the single highest leverage cost control in the observability stack because it taxes data at the door instead of storing regret. Commitment tiers then price the survivors, with the FinOps rhythm watching ingestion by table monthly, the top ten tables report being where every workspace review starts and usually where it finds the accidental gigabytes.

Retention, Archive, and the Long Memory

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

Table Plans and the Door Tax

The cost model rewards sorting telemetry by how it will be read. Analytics tables carry the interactive tier, full KQL, alerts, dashboards, priced accordingly, and reserved for what humans and alert rules actually query. Basic tables take the high volume operational logs queried occasionally and simply, at a fraction of the ingestion price with limited KQL and no alerting. Auxiliary tables take the compliance and forensic flows, cheapest ingestion, queried rarely and slowly, the tier the Sentinel post flagged for the keep everything mandates. Sort honestly, verbose container stdout is rarely Analytics grade, and enforce the sorting upstream with data collection rule transformations: KQL executed at ingestion that drops the debug noise, trims the fat columns, redacts what should never land, and routes rows to the right tables, the single highest leverage cost control in the observability stack because it taxes data at the door instead of storing regret. Commitment tiers then price the survivors, with the FinOps rhythm watching ingestion by table monthly, the top ten tables report being where every workspace review starts and usually where it finds the accidental gigabytes.

Retention, Archive, and the Long Memory

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

The default is one central workspace per region of significance, because correlation is the product, the App Insights trace joining the AKS logs joining the firewall logs works only when they share a workspace, and RBAC happens at finer grains than workspace boundaries anyway. The legitimate exceptions, each argued in earlier posts: the dedicated Sentinel workspace where security’s retention and access diverge hard, regional workspaces where data residency laws pin telemetry, and the rare tenant boundary case for service providers. What the topology should never encode is org chart vanity, a workspace per team recreates the silo problem observability exists to solve, and cross workspace queries are a tax, not a pattern. Access design inside the workspace does the real segregation: resource context access means a team granted read on their resources can query those resources’ logs and no one else’s through the resource’s Logs blade, table level RBAC restricts the sensitive tables (the sign in logs, anything with PII surviving the redaction posts) to the teams with cause, and the granular roles keep the reader and contributor lines clean.

Table Plans and the Door Tax

The cost model rewards sorting telemetry by how it will be read. Analytics tables carry the interactive tier, full KQL, alerts, dashboards, priced accordingly, and reserved for what humans and alert rules actually query. Basic tables take the high volume operational logs queried occasionally and simply, at a fraction of the ingestion price with limited KQL and no alerting. Auxiliary tables take the compliance and forensic flows, cheapest ingestion, queried rarely and slowly, the tier the Sentinel post flagged for the keep everything mandates. Sort honestly, verbose container stdout is rarely Analytics grade, and enforce the sorting upstream with data collection rule transformations: KQL executed at ingestion that drops the debug noise, trims the fat columns, redacts what should never land, and routes rows to the right tables, the single highest leverage cost control in the observability stack because it taxes data at the door instead of storing regret. Commitment tiers then price the survivors, with the FinOps rhythm watching ingestion by table monthly, the top ten tables report being where every workspace review starts and usually where it finds the accidental gigabytes.

Retention, Archive, and the Long Memory

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

The cost model rewards sorting telemetry by how it will be read. Analytics tables carry the interactive tier, full KQL, alerts, dashboards, priced accordingly, and reserved for what humans and alert rules actually query. Basic tables take the high volume operational logs queried occasionally and simply, at a fraction of the ingestion price with limited KQL and no alerting. Auxiliary tables take the compliance and forensic flows, cheapest ingestion, queried rarely and slowly, the tier the Sentinel post flagged for the keep everything mandates. Sort honestly, verbose container stdout is rarely Analytics grade, and enforce the sorting upstream with data collection rule transformations: KQL executed at ingestion that drops the debug noise, trims the fat columns, redacts what should never land, and routes rows to the right tables, the single highest leverage cost control in the observability stack because it taxes data at the door instead of storing regret. Commitment tiers then price the survivors, with the FinOps rhythm watching ingestion by table monthly, the top ten tables report being where every workspace review starts and usually where it finds the accidental gigabytes.

Retention, Archive, and the Long Memory

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

Table Plans and the Door Tax

The cost model rewards sorting telemetry by how it will be read. Analytics tables carry the interactive tier, full KQL, alerts, dashboards, priced accordingly, and reserved for what humans and alert rules actually query. Basic tables take the high volume operational logs queried occasionally and simply, at a fraction of the ingestion price with limited KQL and no alerting. Auxiliary tables take the compliance and forensic flows, cheapest ingestion, queried rarely and slowly, the tier the Sentinel post flagged for the keep everything mandates. Sort honestly, verbose container stdout is rarely Analytics grade, and enforce the sorting upstream with data collection rule transformations: KQL executed at ingestion that drops the debug noise, trims the fat columns, redacts what should never land, and routes rows to the right tables, the single highest leverage cost control in the observability stack because it taxes data at the door instead of storing regret. Commitment tiers then price the survivors, with the FinOps rhythm watching ingestion by table monthly, the top ten tables report being where every workspace review starts and usually where it finds the accidental gigabytes.

Retention, Archive, and the Long Memory

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

The default is one central workspace per region of significance, because correlation is the product, the App Insights trace joining the AKS logs joining the firewall logs works only when they share a workspace, and RBAC happens at finer grains than workspace boundaries anyway. The legitimate exceptions, each argued in earlier posts: the dedicated Sentinel workspace where security’s retention and access diverge hard, regional workspaces where data residency laws pin telemetry, and the rare tenant boundary case for service providers. What the topology should never encode is org chart vanity, a workspace per team recreates the silo problem observability exists to solve, and cross workspace queries are a tax, not a pattern. Access design inside the workspace does the real segregation: resource context access means a team granted read on their resources can query those resources’ logs and no one else’s through the resource’s Logs blade, table level RBAC restricts the sensitive tables (the sign in logs, anything with PII surviving the redaction posts) to the teams with cause, and the granular roles keep the reader and contributor lines clean.

Table Plans and the Door Tax

The cost model rewards sorting telemetry by how it will be read. Analytics tables carry the interactive tier, full KQL, alerts, dashboards, priced accordingly, and reserved for what humans and alert rules actually query. Basic tables take the high volume operational logs queried occasionally and simply, at a fraction of the ingestion price with limited KQL and no alerting. Auxiliary tables take the compliance and forensic flows, cheapest ingestion, queried rarely and slowly, the tier the Sentinel post flagged for the keep everything mandates. Sort honestly, verbose container stdout is rarely Analytics grade, and enforce the sorting upstream with data collection rule transformations: KQL executed at ingestion that drops the debug noise, trims the fat columns, redacts what should never land, and routes rows to the right tables, the single highest leverage cost control in the observability stack because it taxes data at the door instead of storing regret. Commitment tiers then price the survivors, with the FinOps rhythm watching ingestion by table monthly, the top ten tables report being where every workspace review starts and usually where it finds the accidental gigabytes.

Retention, Archive, and the Long Memory

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

Table Plans and the Door Tax

The cost model rewards sorting telemetry by how it will be read. Analytics tables carry the interactive tier, full KQL, alerts, dashboards, priced accordingly, and reserved for what humans and alert rules actually query. Basic tables take the high volume operational logs queried occasionally and simply, at a fraction of the ingestion price with limited KQL and no alerting. Auxiliary tables take the compliance and forensic flows, cheapest ingestion, queried rarely and slowly, the tier the Sentinel post flagged for the keep everything mandates. Sort honestly, verbose container stdout is rarely Analytics grade, and enforce the sorting upstream with data collection rule transformations: KQL executed at ingestion that drops the debug noise, trims the fat columns, redacts what should never land, and routes rows to the right tables, the single highest leverage cost control in the observability stack because it taxes data at the door instead of storing regret. Commitment tiers then price the survivors, with the FinOps rhythm watching ingestion by table monthly, the top ten tables report being where every workspace review starts and usually where it finds the accidental gigabytes.

Retention, Archive, and the Long Memory

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

Table Plans and the Door Tax

The cost model rewards sorting telemetry by how it will be read. Analytics tables carry the interactive tier, full KQL, alerts, dashboards, priced accordingly, and reserved for what humans and alert rules actually query. Basic tables take the high volume operational logs queried occasionally and simply, at a fraction of the ingestion price with limited KQL and no alerting. Auxiliary tables take the compliance and forensic flows, cheapest ingestion, queried rarely and slowly, the tier the Sentinel post flagged for the keep everything mandates. Sort honestly, verbose container stdout is rarely Analytics grade, and enforce the sorting upstream with data collection rule transformations: KQL executed at ingestion that drops the debug noise, trims the fat columns, redacts what should never land, and routes rows to the right tables, the single highest leverage cost control in the observability stack because it taxes data at the door instead of storing regret. Commitment tiers then price the survivors, with the FinOps rhythm watching ingestion by table monthly, the top ten tables report being where every workspace review starts and usually where it finds the accidental gigabytes.

Retention, Archive, and the Long Memory

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

The default is one central workspace per region of significance, because correlation is the product, the App Insights trace joining the AKS logs joining the firewall logs works only when they share a workspace, and RBAC happens at finer grains than workspace boundaries anyway. The legitimate exceptions, each argued in earlier posts: the dedicated Sentinel workspace where security’s retention and access diverge hard, regional workspaces where data residency laws pin telemetry, and the rare tenant boundary case for service providers. What the topology should never encode is org chart vanity, a workspace per team recreates the silo problem observability exists to solve, and cross workspace queries are a tax, not a pattern. Access design inside the workspace does the real segregation: resource context access means a team granted read on their resources can query those resources’ logs and no one else’s through the resource’s Logs blade, table level RBAC restricts the sensitive tables (the sign in logs, anything with PII surviving the redaction posts) to the teams with cause, and the granular roles keep the reader and contributor lines clean.

Table Plans and the Door Tax

The cost model rewards sorting telemetry by how it will be read. Analytics tables carry the interactive tier, full KQL, alerts, dashboards, priced accordingly, and reserved for what humans and alert rules actually query. Basic tables take the high volume operational logs queried occasionally and simply, at a fraction of the ingestion price with limited KQL and no alerting. Auxiliary tables take the compliance and forensic flows, cheapest ingestion, queried rarely and slowly, the tier the Sentinel post flagged for the keep everything mandates. Sort honestly, verbose container stdout is rarely Analytics grade, and enforce the sorting upstream with data collection rule transformations: KQL executed at ingestion that drops the debug noise, trims the fat columns, redacts what should never land, and routes rows to the right tables, the single highest leverage cost control in the observability stack because it taxes data at the door instead of storing regret. Commitment tiers then price the survivors, with the FinOps rhythm watching ingestion by table monthly, the top ten tables report being where every workspace review starts and usually where it finds the accidental gigabytes.

Retention, Archive, and the Long Memory

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

The cost model rewards sorting telemetry by how it will be read. Analytics tables carry the interactive tier, full KQL, alerts, dashboards, priced accordingly, and reserved for what humans and alert rules actually query. Basic tables take the high volume operational logs queried occasionally and simply, at a fraction of the ingestion price with limited KQL and no alerting. Auxiliary tables take the compliance and forensic flows, cheapest ingestion, queried rarely and slowly, the tier the Sentinel post flagged for the keep everything mandates. Sort honestly, verbose container stdout is rarely Analytics grade, and enforce the sorting upstream with data collection rule transformations: KQL executed at ingestion that drops the debug noise, trims the fat columns, redacts what should never land, and routes rows to the right tables, the single highest leverage cost control in the observability stack because it taxes data at the door instead of storing regret. Commitment tiers then price the survivors, with the FinOps rhythm watching ingestion by table monthly, the top ten tables report being where every workspace review starts and usually where it finds the accidental gigabytes.

Retention, Archive, and the Long Memory

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

Table Plans and the Door Tax

The cost model rewards sorting telemetry by how it will be read. Analytics tables carry the interactive tier, full KQL, alerts, dashboards, priced accordingly, and reserved for what humans and alert rules actually query. Basic tables take the high volume operational logs queried occasionally and simply, at a fraction of the ingestion price with limited KQL and no alerting. Auxiliary tables take the compliance and forensic flows, cheapest ingestion, queried rarely and slowly, the tier the Sentinel post flagged for the keep everything mandates. Sort honestly, verbose container stdout is rarely Analytics grade, and enforce the sorting upstream with data collection rule transformations: KQL executed at ingestion that drops the debug noise, trims the fat columns, redacts what should never land, and routes rows to the right tables, the single highest leverage cost control in the observability stack because it taxes data at the door instead of storing regret. Commitment tiers then price the survivors, with the FinOps rhythm watching ingestion by table monthly, the top ten tables report being where every workspace review starts and usually where it finds the accidental gigabytes.

Retention, Archive, and the Long Memory

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

Table Plans and the Door Tax

The cost model rewards sorting telemetry by how it will be read. Analytics tables carry the interactive tier, full KQL, alerts, dashboards, priced accordingly, and reserved for what humans and alert rules actually query. Basic tables take the high volume operational logs queried occasionally and simply, at a fraction of the ingestion price with limited KQL and no alerting. Auxiliary tables take the compliance and forensic flows, cheapest ingestion, queried rarely and slowly, the tier the Sentinel post flagged for the keep everything mandates. Sort honestly, verbose container stdout is rarely Analytics grade, and enforce the sorting upstream with data collection rule transformations: KQL executed at ingestion that drops the debug noise, trims the fat columns, redacts what should never land, and routes rows to the right tables, the single highest leverage cost control in the observability stack because it taxes data at the door instead of storing regret. Commitment tiers then price the survivors, with the FinOps rhythm watching ingestion by table monthly, the top ten tables report being where every workspace review starts and usually where it finds the accidental gigabytes.

Retention, Archive, and the Long Memory

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

The default is one central workspace per region of significance, because correlation is the product, the App Insights trace joining the AKS logs joining the firewall logs works only when they share a workspace, and RBAC happens at finer grains than workspace boundaries anyway. The legitimate exceptions, each argued in earlier posts: the dedicated Sentinel workspace where security’s retention and access diverge hard, regional workspaces where data residency laws pin telemetry, and the rare tenant boundary case for service providers. What the topology should never encode is org chart vanity, a workspace per team recreates the silo problem observability exists to solve, and cross workspace queries are a tax, not a pattern. Access design inside the workspace does the real segregation: resource context access means a team granted read on their resources can query those resources’ logs and no one else’s through the resource’s Logs blade, table level RBAC restricts the sensitive tables (the sign in logs, anything with PII surviving the redaction posts) to the teams with cause, and the granular roles keep the reader and contributor lines clean.

Table Plans and the Door Tax

The cost model rewards sorting telemetry by how it will be read. Analytics tables carry the interactive tier, full KQL, alerts, dashboards, priced accordingly, and reserved for what humans and alert rules actually query. Basic tables take the high volume operational logs queried occasionally and simply, at a fraction of the ingestion price with limited KQL and no alerting. Auxiliary tables take the compliance and forensic flows, cheapest ingestion, queried rarely and slowly, the tier the Sentinel post flagged for the keep everything mandates. Sort honestly, verbose container stdout is rarely Analytics grade, and enforce the sorting upstream with data collection rule transformations: KQL executed at ingestion that drops the debug noise, trims the fat columns, redacts what should never land, and routes rows to the right tables, the single highest leverage cost control in the observability stack because it taxes data at the door instead of storing regret. Commitment tiers then price the survivors, with the FinOps rhythm watching ingestion by table monthly, the top ten tables report being where every workspace review starts and usually where it finds the accidental gigabytes.

Retention, Archive, and the Long Memory

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

Every diagnostic setting in this series pointed at a Log Analytics workspace, and the accumulated flow is now the estate’s second memory and, unmanaged, one of its top line items. Today is the architecture of that memory: how many workspaces, which table plans, what transforms at the door, and who reads what.

Workspace Topology

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

The cost model rewards sorting telemetry by how it will be read. Analytics tables carry the interactive tier, full KQL, alerts, dashboards, priced accordingly, and reserved for what humans and alert rules actually query. Basic tables take the high volume operational logs queried occasionally and simply, at a fraction of the ingestion price with limited KQL and no alerting. Auxiliary tables take the compliance and forensic flows, cheapest ingestion, queried rarely and slowly, the tier the Sentinel post flagged for the keep everything mandates. Sort honestly, verbose container stdout is rarely Analytics grade, and enforce the sorting upstream with data collection rule transformations: KQL executed at ingestion that drops the debug noise, trims the fat columns, redacts what should never land, and routes rows to the right tables, the single highest leverage cost control in the observability stack because it taxes data at the door instead of storing regret. Commitment tiers then price the survivors, with the FinOps rhythm watching ingestion by table monthly, the top ten tables report being where every workspace review starts and usually where it finds the accidental gigabytes.

Retention, Archive, and the Long Memory

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

Table Plans and the Door Tax

The cost model rewards sorting telemetry by how it will be read. Analytics tables carry the interactive tier, full KQL, alerts, dashboards, priced accordingly, and reserved for what humans and alert rules actually query. Basic tables take the high volume operational logs queried occasionally and simply, at a fraction of the ingestion price with limited KQL and no alerting. Auxiliary tables take the compliance and forensic flows, cheapest ingestion, queried rarely and slowly, the tier the Sentinel post flagged for the keep everything mandates. Sort honestly, verbose container stdout is rarely Analytics grade, and enforce the sorting upstream with data collection rule transformations: KQL executed at ingestion that drops the debug noise, trims the fat columns, redacts what should never land, and routes rows to the right tables, the single highest leverage cost control in the observability stack because it taxes data at the door instead of storing regret. Commitment tiers then price the survivors, with the FinOps rhythm watching ingestion by table monthly, the top ten tables report being where every workspace review starts and usually where it finds the accidental gigabytes.

Retention, Archive, and the Long Memory

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

Table Plans and the Door Tax

The cost model rewards sorting telemetry by how it will be read. Analytics tables carry the interactive tier, full KQL, alerts, dashboards, priced accordingly, and reserved for what humans and alert rules actually query. Basic tables take the high volume operational logs queried occasionally and simply, at a fraction of the ingestion price with limited KQL and no alerting. Auxiliary tables take the compliance and forensic flows, cheapest ingestion, queried rarely and slowly, the tier the Sentinel post flagged for the keep everything mandates. Sort honestly, verbose container stdout is rarely Analytics grade, and enforce the sorting upstream with data collection rule transformations: KQL executed at ingestion that drops the debug noise, trims the fat columns, redacts what should never land, and routes rows to the right tables, the single highest leverage cost control in the observability stack because it taxes data at the door instead of storing regret. Commitment tiers then price the survivors, with the FinOps rhythm watching ingestion by table monthly, the top ten tables report being where every workspace review starts and usually where it finds the accidental gigabytes.

Retention, Archive, and the Long Memory

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

The default is one central workspace per region of significance, because correlation is the product, the App Insights trace joining the AKS logs joining the firewall logs works only when they share a workspace, and RBAC happens at finer grains than workspace boundaries anyway. The legitimate exceptions, each argued in earlier posts: the dedicated Sentinel workspace where security’s retention and access diverge hard, regional workspaces where data residency laws pin telemetry, and the rare tenant boundary case for service providers. What the topology should never encode is org chart vanity, a workspace per team recreates the silo problem observability exists to solve, and cross workspace queries are a tax, not a pattern. Access design inside the workspace does the real segregation: resource context access means a team granted read on their resources can query those resources’ logs and no one else’s through the resource’s Logs blade, table level RBAC restricts the sensitive tables (the sign in logs, anything with PII surviving the redaction posts) to the teams with cause, and the granular roles keep the reader and contributor lines clean.

Table Plans and the Door Tax

The cost model rewards sorting telemetry by how it will be read. Analytics tables carry the interactive tier, full KQL, alerts, dashboards, priced accordingly, and reserved for what humans and alert rules actually query. Basic tables take the high volume operational logs queried occasionally and simply, at a fraction of the ingestion price with limited KQL and no alerting. Auxiliary tables take the compliance and forensic flows, cheapest ingestion, queried rarely and slowly, the tier the Sentinel post flagged for the keep everything mandates. Sort honestly, verbose container stdout is rarely Analytics grade, and enforce the sorting upstream with data collection rule transformations: KQL executed at ingestion that drops the debug noise, trims the fat columns, redacts what should never land, and routes rows to the right tables, the single highest leverage cost control in the observability stack because it taxes data at the door instead of storing regret. Commitment tiers then price the survivors, with the FinOps rhythm watching ingestion by table monthly, the top ten tables report being where every workspace review starts and usually where it finds the accidental gigabytes.

Retention, Archive, and the Long Memory

Retention splits into interactive and total: the interactive window (per table, override the workspace default per need) keeps data hot for queries and alerts, and the archive extension keeps it cheap for years beyond, reachable by search jobs and restore operations when the auditor or the incident from eighteen months ago calls. Set the splits by table category, ninety interactive days for the operational core, shorter for the verbose, long archive for the security and audit tables per the compliance calendar, and export the truly regulated flows to the immutable storage tiers from the September posts where the mandate demands storage you can prove untouched. The design deliverable this post argues for is a one page telemetry register, table, source, plan, transform, retention, owner, reviewed quarterly like the policy exemptions, because the workspace is a data platform now, and data platforms run on exactly that discipline. Tomorrow the governance lens widens from telemetry to all data with Microsoft Purview.

Cheers
Osama

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.