Module

IT Inventory & Managed Assets

Reorganizing the flat IT Admin card grid into a containment model — servers hold websites and databases; the API website owns its configurable functions — so every IT record sits behind the asset it belongs to.

Status Draft
Last updated
Related PlatformWorkflow Engine
OverviewSystem Map

Overview

The IT Admin dashboard (it-admin-page) currently renders a flat grid of cards — Server Jobs, SQL Entities, Websites, Webhooks, File Watcher, Server Activity, Conversation Bots, API Docs — grouped only by loose sections (IT Inventory, IT Health, IT Development). Nothing tells you which server or site a given record runs on; the relationships exist in the data but the UI is a bag of peers.

The it.Server table already exists and holds the real assets (a web server row, a database server row), but there is no server list or detail page in the portal — the table is only used to populate a dropdown on Server Jobs. Confusingly, the module named server-inventory is not a server registry at all; it is the API activity-logs + IP-blocking module (see IA).

The goal of this initiative: turn IT Admin into a containment model of managed assets. You navigate to a server, see the websites and databases it hosts; navigate into the API website, see the functions it runs (watchers, webhooks, bots, activity, docs); navigate into a database, see its schemas and tables. Every current card becomes a view behind the asset it belongs to.

Two Kinds of Relationship

The organizing insight — and the answer to "which records get a foreign key" — is that the IT records relate to assets in two fundamentally different ways. Conflating them is what produces meaningless keys.

KindTestRepresentationExamples
Placement link The child could genuinely live on a different parent — the value varies row to row. Real FK (column + navigation property + configured EF relationship). A website runs on this server vs that one; a database on this server; a job scheduled on this host; a log came from a specific host or site.
Intrinsic function The record could only ever belong to the one API app — it is a configurable feature of that codebase. No FK. Grouped under the API website by definition, not by a stored pointer. File watchers, webhooks, conversation bots — these are the API's functions. They cannot be "moved" to another website.
Why intrinsic functions get no key. A WebsiteId on FileWatcherJob, Webhook, or BotConversation would be a column that is always the API website's id and can never be anything else — noise, not a relationship. These features aren't hosted on the API website the way a website is hosted on a server; they are functions of the API code itself. They nest under "the API" in the UI because that is the only place they can be, so the grouping is resolved by identifying the API website (below), not by joining a foreign key.

The Containment Model

Two tiers of containment. Server is the physical/virtual host; Website is an app hosted on a server. Most "API-tied" features hang off the API website (which itself hangs off the web server) — a two-hop path, not a direct server link.

Server (Role = Web) ──┬── Website (Role = Public) ── CMS Web Pages
                      │
                      └── Website (Role = Api, IsHost) ──┬── File Watchers   ┐
                                                         ├── Webhooks        │ intrinsic
                                                         ├── Conversation Bots│ functions
                                                         ├── Activity Logs   │ (no FK —
                                                         └── API Docs / OpenAPI┘ by definition)

Server (Role = Database) ── ServerDatabase ── ServerSchema ── ServerTable / ServerView   (= SQL Entities)

Server-scoped (placement FK to Server, not to a website):
      ServerJob (runs on a server)   ·   ServerLog (server events)

The API application has no entity of its own — it is it.Website row 11. That is convenient: "everything tied to the API" resolves to "the website identified as the API host," and its intrinsic functions render as that site's feature tabs.

Anchor Entities

Server — it.Server

PK Id. Fields: Name (FQDN), OperatingSystem, IPAddress, CompassComponentId (→ atlassian.CompassComponent), JSON Data. Navigations: Websites, ServerDatabases.

New — Server.Role. There is currently no column distinguishing a web server from a database server; the distinction is only inferred from which children point at it. Add an explicit Role (Web, Database; nullable — a box that is both is the rare exception) so the dashboard classifies a server without inference.

Website — it.Website

PK Id. Placement FK to Server: already wired (ServerId, required, FK_Website_Server). Fields: Name, Binding (IIS host, e.g. www.auburnal.gov), RootPath, LogPath, State, JSON Data, VDirs. Also the root of the CMS Webpage hierarchy (FK_Webpage_Website).

New — Website.Role + Website.IsHost. Mirroring the server, Role classifies the site (Api, Public/Cms, …) and drives which feature tabs it surfaces — an Api-role site shows Watchers / Webhooks / Bots / Activity / Docs; a public site does not. IsHost (bool) marks the canonical running instance (the self record). "The API" = the row where Role = Api AND IsHost = true. Role is the classification; IsHost disambiguates which Api-role row is the live self when more than one exists (environment mirrors, a future second surface). This replaces the hardcoded WebsiteId = 11 scattered in code with a single self-describing lookup.
Exactly one host. The resolver for "the API website" must treat Role = Api AND IsHost as returning one row — pick deterministically and guard against multiple matches — so nothing silently grabs the wrong site.

Keys Audit

What links exist today, and the gap. Only placement links appear here; intrinsic functions are deliberately keyless.

EntityLinkStatusAction
Website → ServerServerIdWired
ServerDatabase → ServerServerIdWired
ServerSchema / ServerTable / ServerViewvia DB → Schema chainWired— (indirect, correct)
ServerJob → ServerServerId (nullable)Loose columnPromote to a real EF relationship + nav property.
ServerLog → ServerServerId (required)Loose columnPromote to a real EF relationship + nav property.
WebsiteLog → WebsiteWebsiteId (default 11)Loose columnPromote to a real EF relationship; source the default from the API-host lookup, not a literal.
FileWatcherJob, Webhook, BotConversationIntrinsicNo FK. Grouped under the API host by resolver. Webhooks additionally live in the organization schema.
FK promotion is split from the additive columns — deliberately. The role/host columns (Server.Role, Website.Role, Website.IsHost) are purely additive (nullable, or non-null with a default) and ship as a zero-risk migration that unblocks the drill-down UI. Promoting the three loose id columns to real EF relationships is deferred to a follow-up because adding a DB foreign-key constraint validates every existing row — and two of the three targets are log tables. WebsiteLog especially is a high-volume, append-only table hardcoded to WebsiteId = 11; a constraint there is real operational cost (scan + lock on apply) for little gain, since the UI resolves "logs owned by the API" through the IsHost lookup rather than a join. The portal filters on the existing scalar id columns and does not need the constraints. Promote them after a quick orphaned-row audit, log tables last (or never).

Information Architecture

Collapse the flat card grid into two drill-downs rooted on the anchor entities. The existing pages mostly survive — they get re-parented behind Server / Website detail instead of being top-level cards, and each list gains a server/website scope filter.

Servers module (new)

  • Server list — the missing page. Rows show Role, OS, IP.
  • Server dashboard — stat cards for the hosted assets (Websites, Databases, Jobs), the hosted-website list as the primary view, and a sidebar with the server's identity (edit-in-drawer sets Role) plus database/job summaries. The stat-card layout doubles as the relationship view and the drill-down; a plain tab strip showed the relationships but drilled poorly.

Website detail

  • A per-website dashboard. For an Api-role site it surfaces the request-traffic KPIs (the current server-inventory activity dashboard — WebsiteLog is per-website data), a Recent Errors panel (this is the real home for ServerLog, which is effectively an API-app error log), and the site's intrinsic functions: File Watchers, Webhooks, Conversation Bots, API Docs. These are the current IT-Inventory / IT-Health cards, re-homed as the API website's functions. A public site gets a simpler view (bindings, pages, state).
  • For a Public-role site: CMS web pages, bindings, state — no function tabs.

Database drill-down

  • The server dashboard's stat cards are clickable (Websites/Databases scroll to their list; Jobs opens the server-jobs page), and Databases is a drillable list in the main column. From there the full catalog chain is navigable: Server → Database → Schema → Table/ViewDatabaseDetailPage lists a ServerDatabase's schemas, SchemaDetailPage lists a ServerSchema's tables and views (the leaf). This supersedes the flat /admin/sql-entities list (all views/tables un-scoped). Each level's Data JSON is surfaced generically as key/value rows.
  • Table usage stats answer "is this object utilized?" The catalog job now captures per-table row count / size (sys.dm_db_partition_stats) and last write / last read (sys.dm_db_index_usage_stats), promoted to real ServerTable columns so "biggest / never-read tables" are sortable. Views (no storage) get last-modified + a referenced-by count (sys.sql_expression_dependencies) as a static-utilization proxy. Because the index-usage DMV resets on SQL restart, last-read/last-write are accumulated as a running MAX across catalog runs — the recurring job turns a volatile snapshot into a durable last-used date. Real view runtime usage (Query Store) and a stale-object soft-delete are v2.
Naming collision to resolve. The module folder my/js/modules/server-inventory/ is actually the API activity-logs + IP-registry module, not a server registry. Rename it to what it is (e.g. api-activity) so the accurate name is free for the real servers module, and so the activity views can slot cleanly under the API website's Activity tab.

Server Jobs & SQL Agent Sync

The it.ServerJob table acts as the configuration source of truth and metadata cache for tasks that execute as Microsoft SQL Server Agent jobs. Because execution happens in the SQL Agent, state is synchronized bi-directionally between the application database and the msdb system database.

Table Schema & SQL Agent Mapping

FieldDescriptionSQL Agent Representation
Name / DescriptionIdentity and human-readable context.Mapped directly to the SQL Agent Job's Name and Description.
ScheduleCron expression determining when the job runs.Parsed into a SQL Agent Schedule (sp_add_schedule) and attached to the job.
CodeThe execution script or instruction.Forms the command text of the job's single step (Step 1).
IsEnabledToggles whether the job is active.Mapped to the @enabled flag on the SQL Agent job.
LastRun / NextRunTimestamps of the most recent and upcoming executions.Synced back from sysjobservers.last_run_date/time and sysjobschedules.next_run_date/time.
LastRunStatus / LastRunMessageOutcome details of the last execution (e.g., Success, Failed, Running).Synced back from the SQL Agent job outcome (0=Failed, 1=Success) or set by the API during execution.

Synchronization Triggers

Data moves between the application and SQL Agent through two primary pathways:

  • App ➔ SQL Agent (Synchronous Push): Whenever a job is created, updated, deleted, enabled, or disabled via the API, the changes are instantly pushed to the SQL Agent via ISqlAgentService.UpsertSqlAgentJobAsync. The background service also ensures definitions stay aligned every 15 minutes.
  • SQL Agent ➔ App (Metadata Pull): A background worker (ServerJobSyncService) polls the SQL Agent every 15 minutes to fetch the latest LastRun, NextRun, and outcome status for all jobs. Additionally, when a workflow job is executed manually via the API, the API synchronously fetches the updated NextRun immediately upon completion.
Pure T-SQL Jobs and Stale Fields. If the Code field begins with standard T-SQL commands (like USE, SELECT, or EXEC), the sync engine configures the SQL Agent job subsystem as TSQL instead of CmdExec (which normally wraps the execution in a PowerShell script that calls the API). Because pure T-SQL jobs execute entirely on the SQL server without invoking the API's WorkflowActionRunner, the API is blind to their execution in real-time. Consequently, fields like LastRun, NextRun, and LastRunStatus will remain stale until the 15-minute background polling cycle catches them up.

IIS Inventory & Per-Website Logs

Web-server sites are inventoried and their access logs captured by a single workflow ServerJob. This section covers that pipeline and how per-website traffic and certificate data reach the dashboards.

The inventory pipeline

One ServerJob (WebDevelopment.IT.IIS-Discovery), actions in order:

#ActionDoes
1media-stats scriptCMS media rollups (out of scope here).
2Get-IIS-Inventory.ps1 (NetworkConnection · impersonated on the web server)Emits JSON of every site (bindings, paths, state, certs, log config, app pool); a following MsSql Merge upserts it into it.Website.
3Audit-IISLogs.ps1LogParser reads each site's W3C logs → bulk-inserts into cms.IISLogEntry.
4IISLogParseHandler opsmatch-contentupdate-media-downloadsrefresh-dependencies: links log rows to CMS pages/media/aliases and rolls up download counts.
5media-stats sprocFinal CMS stat rollup.

The ps1 parses, the handler post-processes — complementary, not double-parsing (the handler's own parse-logs action is dormant in this flow).

The merge is a generic column intersection. The MsSql Merge upserts the intersection of the script's JSON keys and the target table's columns (join key Name). Two consequences drive the design below: a new top-level scalar in the script output auto-maps the moment a matching column exists (no config change), and nested lists (the script's Bindings) map to nothing — so per-binding data has to ride inside a column that does exist (Data).

Three log sources — don't conflate them

SourceWhatScopeKeyed by
it.WebsiteLogthe API's own request-middleware logthe API site only (WebsiteId = 11)WebsiteId
cms.IISLogEntryparsed IIS W3C access logsall sitesSiteName string → now also WebsiteId
ApiActivity*Dailynightly rollups of WebsiteLogthe API site

The per-website dashboards are powered by IISLogEntry (every site); the API site additionally has the richer WebsiteLog view. Before this work, IISLogEntry had no read path — which is why every site but the API showed a blank page.

Per-website logs

  • Site link. IISLogEntry was keyed only by the SiteName string. Added a nullable, indexed WebsiteIda plain column, not an enforced FK, for the same reason as the loose log-table columns above (a constraint on a high-volume log table is operational cost for little gain). The parser stamps it on new rows; the existing ~2.5M rows were backfilled from SiteName = Website.Name.
  • Read path. GET /it/website/{id}/iis-logs returns recent rows plus a small aggregate (status mix, bot share, top paths), matched by WebsiteId with a SiteName fallback for the transition window. The website dashboard shows a traffic panel (tiles + Recent Requests + Top Paths) for every site.
Backfills belong in a script, not the migration. The first cut put the WebsiteId backfill inside the EF migration as a Sql() join-UPDATE. On the ~2.5M-row table it blew past EF's 30-second migration CommandTimeout, and because the migration runs in a transaction the whole thing rolled back — nothing applied. Rule: schema in the migration, data in a separate batched script (db/backfill-iislog-websiteid.sql, UPDATE TOP (n) … WHILE @@ROWCOUNT > 0) run outside that timeout.

Expanded site info & certificates

Get-IIS-Inventory.ps1 now also captures, per site: per-binding certificate detail (thumbprint → subject / issuer / NotAfter / DNS names, resolved from the local store), log configuration (format / period / enabled / truncate), and app-pool info (CLR / pipeline / identity / state) — all folded into the Data JSON column.

Cert expiry is a promoted scalar, not a JSON field. A site's earliest certificate expiry is written to a dedicated it.Website.EarliestCertExpiry column (min NotAfter across bindings) rather than left inside the Data/Binding blob — so "certificates expiring in N days" is a cheap, sortable, alertable query. It auto-maps through the generic merge because a matching column now exists. Per-binding cert detail stays in Data (which is merged); the full per-binding WebsiteBinding table (dropped Jan 2026 in favour of JSON) is a v2 normalization if binding-level querying is ever needed.
Latent inconsistency, flagged not fixed. Because the merge maps by column name, the script's Bindings list does not populate the singular it.Website.Binding column — yet the portal still parses Binding for its URLs, so how that column is currently populated is unclear. Worth resolving when the binding pipeline is next touched.

Deferred to v2

  • One ingest path. Retire Audit-IISLogs.ps1 in favour of the C# handler's parse-logs (in-process, no external LogParser, already does the CMS linkage) — keeping both is redundant once WebsiteId stamping is everywhere.
  • IIS control from the API. A dedicated PowerShellHandler workflow action (start / stop / recycle a site), modelled on NetworkConnectionHandler.execute (impersonated pwsh under a credential). Turns Website.State from daily-informational into an action, and enables on-demand inventory refresh. Guardrails: Admin policy, confirm-before-stop, audit. The API runs on the web server, so this is feasible without elevating its app-pool identity.
  • In-app Role/IsHost editing. A PATCH /it/website/{id} endpoint (the controller is GET-only today), so the website identity card's edit drawer can set them instead of SQL.

Phased Plan

PhaseWorkNotes
1a — Role/host columns Add Server.Role, Website.Role, Website.IsHost + an "API host website" resolver (GetApiHostWebsiteAsync, one-row guarantee). Backfill Role/IsHost on the existing rows. Additive, zero-risk migration. No UI. Unblocks the drill-downs. Migration staged
1b — FK promotion (deferred) Promote ServerJob.ServerId, ServerLog.ServerId, WebsiteLog.WebsiteId to real EF relationships + nav props, after an orphaned-row audit. Log tables last. Adds DB constraints → row validation on apply. Not required by the portal. See the Keys Audit note.
2 — Servers module Server list + Server dashboard — stat cards (Websites / Databases / Jobs counts) + hosted-website list + identity/edit sidebar. Scopes by the existing scalar serverId (dedicated per-server endpoints; jobs filtered client-side). ServerLog moved to the website level (Phase 3). Fills the missing page; the anchor of the whole model. Built
3a — Website dashboard Per-website dashboard reached from the server's hosted-website list: identity (URLs / state / role / API-host / server backlink / paths) + a function-card grid (Watchers / Webhooks / Bots / API Docs / Activity) shown for the resolved API host. Reads Role/IsHost (SQL-backfilled). Makes website rows drillable; re-homes the API's functions under it. Built
3b — KPIs, errors, edit, rename Inline API request KPIs (scope the activity endpoints by website id) + a Recent Errors panel (fix ServerLog.TimestampDatabaseGenerated(Identity) with no default inserts NULL; stamp UtcNow + UTC-on-read). Add PATCH /it/website/{id} so Role/IsHost are editable in-app. Rename server-inventoryapi-activity. The flat IT-Inventory / IT-Health cards retire into drill-downs.
4 — Dashboard Replace the flat IT Admin card grid with an assets overview (servers, their sites/dbs, health rollups) as the entry point. Design-level TBD; depends on 2–3.
IIS — inventory & logs Per-website IIS-log endpoint + traffic panel; IISLogEntry.WebsiteId + backfill; Website.EarliestCertExpiry; enriched inventory script (certs / log config / app pool). See IIS Inventory & Logs. Per-website logs + schema Built; enrichment scripts done, deploy pending. IIS control & PATCH → v2.

Decisions Settled

  • Two-tier grain. API-tied features relate to the API website, and the website relates to the server — a two-hop path. Features do not carry a direct ServerId.
  • Intrinsic functions carry no FK. File watchers, webhooks, and bots are functions of the API code; a WebsiteId on them would be a constant, so it is omitted. They are grouped by resolving the API host, not by a join.
  • Server and Website both get a Role; Website additionally gets IsHost. "The API" = Role = Api AND IsHost, resolved to exactly one row — replacing the hardcoded WebsiteId = 11.
  • Keys before UI. Phase 1 wires the relationships; the drill-down pages are straightforward once the model is real.
  • Log-table site links are plain indexed columns, not FKs. IISLogEntry.WebsiteId (like the deferred WebsiteLog/ServerLog links) is a nullable indexed column — a constraint on a high-volume log table isn't worth the write/validation cost.
  • Cert expiry is promoted to a column. Website.EarliestCertExpiry makes "expiring soon" queryable; per-binding detail stays in the merged Data JSON.
  • Data backfills never live in an EF migration. Schema in the migration, backfill in a separate batched script — EF's 30s migration timeout + transaction turns a heavy UPDATE into a full rollback.
  • Keep both IIS parsers in v1. The ps1 parses and the C# handler post-processes (CMS linkage); consolidating onto one path is a v2 cleanup, not a prerequisite.