← Back to docs

Session Observability E2E Testbook

Manual verification of session page features: Sunshine unreachable badge (v0.4.17), session log viewer (v0.4.17), head name column and conditional refresh (v0.4.18), orphaned stream dashboard KPI and Active Problems panel (v0.4.20), expandable body rows with inline session detail and Self-Healing panel (v0.4.22), Stream page provider_status error badges on body cards (v0.4.23).

Prerequisites

Test 1: Sunshine unreachable badge

Goal: When a body node reports provider_status: "sunshine_api_unreachable", the active sessions table shows an amber warning badge next to the body name. The badge disappears when Sunshine recovers.

Steps

  1. Navigate to https://hydrastreamingmonitor.experiencenet.com/sessions.

  2. Start a stream from a head to a body so an active session appears in the table.

  3. Stop the Sunshine service on the body to trigger the unreachable status (from a terminal with hydracluster access):

    curl -sf -X POST -H "Authorization: Bearer $HYDRACLUSTER_TOKEN" \
  "$HC_SERVER/api/v1/nodes/$BODY_NODE/exec" \
  -H "Content-Type: application/json" \
  -d '{"command":"powershell","args":["-Command","Stop-Service -Name Sunshine -Force"],"timeout_seconds":30}'

  4. Wait up to 60 s for the body to report the unreachable status, then reload the sessions page (or wait for the auto-refresh — 5 s while an active session is present).

    Expected: The body name cell in the active sessions table shows an amber ⚠ Sunshine unreachable badge.

  5. Restart Sunshine on the body, wait one body tick (~30 s), then reload the sessions page.

    Expected: The badge is gone; the body name cell shows only the name.

Pass criteria

Test 1b: Stream page — provider_status error badge on body cards

Goal: When a body node reports a non-healthy provider_status (e.g. sunshine_api_unreachable), the /stream page shows an amber provider error badge and warning message on the body card instead of the normal idle state. The card returns to normal when Sunshine recovers.

Steps

  1. Navigate to https://hydrastreamingmonitor.experiencenet.com/stream.

  2. Confirm that all healthy idle bodies show a grey idle badge and the text "Ready — no active session".

  3. Stop the Sunshine service on a body to trigger the unreachable status:

    curl -sf -X POST -H "Authorization: Bearer $HYDRACLUSTER_TOKEN" \
  "$HC_SERVER/api/v1/nodes/$BODY_NODE/exec" \
  -H "Content-Type: application/json" \
  -d '{"command":"powershell","args":["-Command","Stop-Service -Name Sunshine -Force"],"timeout_seconds":30}'

  4. Wait up to 60 s for the body to report the unreachable status, then reload the stream page (or wait for the 30 s auto-refresh).

    Expected: The body card for that node shows:

    • An amber provider error badge (replacing the grey idle badge)
    • An amber left border on the card
    • An amber warning message: ⚠ Provider unreachable — sunshine_api_unreachable
    • No "Ready — no active session" text

  5. Restart Sunshine on the body, wait one body tick (~30 s), then reload the stream page.

    Expected: The card returns to normal — grey idle badge, grey left border, "Ready — no active session" text.

Pass criteria

Test 2: Session logs page

Goal: Each row in the active and history session tables has a "logs" link. Clicking it opens a page showing body logs and head logs for that session's time window.

Steps

  1. Navigate to https://hydrastreamingmonitor.experiencenet.com/sessions.

  2. Verify a "logs" column is present in both the active sessions table and the session history table.

  3. Start a stream and let it run for at least 30 s so there is log content. End the stream and wait for it to appear in history.

  4. Click the "logs" link on the completed history row.

    Expected: A session logs page opens showing:

    • Session metadata: session ID, body name, head ID (if known), start/end times
    • Two log panels: "Body logs" and "Head logs" with scrollable <pre> blocks
    • Body logs contain lines with YYYY/MM/DD HH:MM:SS timestamp prefixes within the session window
    • Head logs contain lines if the head node was reachable during the session (may be empty if head log endpoint unavailable)

  5. Click the "logs" link on an active session (if one is running).

    Expected: Same page but with no end time; body logs up to the current moment.

  6. Navigate to /sessions/nonexistent/logs.

    Expected: A 404 page or "session not found" message.

Pass criteria

Test 3: Head name column (v0.4.18)

Goal: The Head column in both active and history session tables shows the human-readable head node name (e.g. ipad-head-2) rather than the raw node ID.

Steps

  1. Navigate to https://hydrastreamingmonitor.experiencenet.com/sessions.

  2. Start a stream from a known head (e.g. ipad-head-2) to a body.

  3. Check the Head column in the active sessions table.

    Expected: The cell shows the head's name (ipad-head-2), not the raw node ID (node-ea866a02).

  4. End the stream and wait for it to appear in session history.

  5. Check the Head column in the history table.

    Expected: Same head name shown for the completed session row.

Pass criteria

Test 4: Conditional auto-refresh (v0.4.18)

Goal: The page refreshes every 5 s during active sessions and every 30 s when idle.

Steps

  1. Navigate to https://hydrastreamingmonitor.experiencenet.com/sessions with no active session.

    Expected: Page footer reads Auto-refreshes every 30s. View-source confirms <meta http-equiv="refresh" content="30">.

  2. Start a stream so an active session appears.

  3. Navigate to or reload the sessions page.

    Expected: Page footer reads Auto-refreshes every 5s (active session). View-source confirms <meta http-equiv="refresh" content="5">.

  4. End the stream and wait for the active sessions table to be empty (one 5 s refresh cycle).

  5. Reload the page.

    Expected: Footer returns to Auto-refreshes every 30s.

Pass criteria

Test 5: Orphaned stream observability (v0.4.22)

Goal: When a body is streaming with no head connected, the dashboard shows an orange "Orphaned Streams" KPI card and the /bodies page shows a "Self-Healing" panel with auto-resolve estimates and Force Stop / File Issue buttons. Body rows expand inline to reveal session detail, recent cleanup history, and GPU watchdog state.

Prerequisites

Steps

  1. With an orphaned body present, navigate to the dashboard: https://hydrastreamingmonitor.experiencenet.com/

    Expected: The 4th KPI card reads "Orphaned Streams" with a non-zero count in orange. An alert banner appears below the KPI row reading "N body/ies stuck in streaming with no head connected" with a link to /bodies.

  2. Click the /bodies link (from the banner or the nav).

    Expected: A blue "Self-Healing (N)" panel appears above the body table. Each orphaned body appears as a card with:

    • Body name in bold
    • District, venue, and auto-resolve estimate (~1 min for grace period, ~3 min for GPU mismatch)
    • A blue resolve badge showing the expected time to auto-clear
    • A Force Stop button
    • A File Issue button (only when hydraissue_token is configured)

  3. In the table, tap/click an orphaned body's row.

    Expected: The row expands to show an inline detail panel with:

    • Active Session block: app name, duration, and state chip (active/disconnected)
    • GPU Watchdog block (only when gpu_mismatch_sec > 0): seconds elapsed and expected kill time
    • Recent Cleanups block: chronological list of cleanup events with action and reason

  4. Check the always-visible Actions column on the orphaned body's row (visible without expanding).

    Expected: The Actions column shows View Logs, Force Stop, and Create Issue buttons directly in the row.

  5. Click Force Stop on an orphaned body (from the Actions column or the Self-Healing panel).

    Expected: Page redirects to /bodies with a green flash notice (or no notice — redirect is immediate). After the next 30 s body heartbeat cycle, the body shows as idle in the table and disappears from the "Self-Healing" panel.

  6. Create a new orphaned body (or use a second one). Click Create Issue in the Actions column.

    Expected: Page redirects to /bodies with a green flash reading "Issue created on issues.experiencenet.com". A new issue appears on issues.experiencenet.com with title "Orphaned stream: {body-name}" and the body name, district, venue, and node ID in the description.

  7. With no orphaned bodies, navigate to the dashboard.

    Expected: The "Orphaned Streams" KPI card shows 0 in muted color (not orange). No alert banner is shown.

  8. Open the /bodies page with no orphaned bodies.

    Expected: No "Self-Healing" panel is visible above the table.

  9. Tap/click a non-orphaned streaming body row.

    Expected: Row expands showing the active session app name, duration, and "active" chip in the Active Session block. Recent Cleanups block shows if any cleanups occurred in the last 24 h. No inline Actions block in the expanded detail (buttons are in the always-visible Actions column instead).

  10. Click View Logs in the Actions column on any online body (not streaming, i.e. no active session).

    Expected: Navigates to /bodies/{nodeID}/logs showing a full-page body log view with the body name as title, all available log lines in a scrollable <pre> block, and a back link to /bodies.

Pass criteria

Test 6: Unified log view (v0.4.22)

Goal: /logs shows body logs, session events, and anomalies merged by timestamp. Source and body filters hide/show rows without a page reload.

Steps

  1. Navigate to https://hydrastreamingmonitor.experiencenet.com/logs.

    Expected: Page loads with a filter bar (Source checkboxes: Body Logs, Sessions, Anomalies; Body checkboxes: one per body that has entries) and a table of log entries sorted newest first. Entry count is shown above the table.

  2. Verify each source type appears with the correct badge colour:

    • Body Logs: blue body badge — lines from hydrabody.log
    • Sessions: green session badge — "session started: ..." / "session ended: ... (duration N)"
    • Anomalies: orange anomaly badge — "orphan_stream: ..." or "relaunch: ..."

  3. Uncheck Body Logs in the Source filter.

    Expected: All rows with a blue body badge disappear immediately (no page reload). Entry count updates.

  4. Re-check Body Logs, then uncheck one specific body in the Body filter.

    Expected: All rows for that body (across all source types) disappear. Rows for other bodies are unchanged.

  5. Uncheck All in the Body filter.

    Expected: All body-specific rows hide. Entry count may reach 0 if no body-agnostic entries exist.

  6. Re-check All, then start a stream and wait up to 30 s for the page to auto-refresh.

    Expected: A new session row appears near the top reading "session started: {experience}". The entry count increases.

  7. End the stream and wait for the next auto-refresh.

    Expected: A new session row appears reading "session ended: {experience} (duration Ns)".

  8. On mobile (or narrow viewport): verify the filter bar wraps to multiple lines without horizontal overflow, and that the log table is readable without side-scrolling.

Pass criteria