Monitor Your Docker Containers from the Command Line with Stacker

What You'll Learn

How to keep an eye on your running Docker containers using nothing but the stacker CLI — check health, stream logs, read deploy events, restart a stuck container, and even hand your logs to an AI for a diagnosis. No Grafana to run, no Portainer to expose, no browser open. Just your terminal and an SSH-native workflow.

• See live container state (CPU, memory, up/down) remotely
• Stream and filter logs from the command line
• Find out why a deploy paused or a container crashlooped
• Feed logs to an AI — including a fully local, private model
• Script it all into alerts, aliases, and cron jobs

Quick Answer

stacker status                 # is the deployment up? IP, services, health
stacker agent health           # live per-container CPU% / MEM% / state
stacker logs -f --tail 100     # stream container logs
stacker deployment events      # why did it pause / fail?
stacker agent logs app --limit 200 | stacker ai ask "why is this crashing?"

Why Monitor From the CLI?

A web dashboard is another service you have to run, expose, secure, and keep updated — just to answer "is my container healthy?" On a small server that's a lot of overhead for a yes/no question. The CLI is SSH-native, works over a slow link, returns machine-readable JSON you can pipe into scripts, and never opens a port you didn't already need. And because the answers are text, you can hand them straight to an AI.

The Mental Model

A Stacker deployment is a set of services (containers) on a server. A small Status Panel agent runs alongside them and answers CLI queries over the message bus — so stacker agent … reads live container state without a browser or an exposed dashboard. Two families of commands:

• stacker status / stacker deployment … — deployment-level state from the API
• stacker agent … — live container health, logs, and control, straight from the agent on the box

Step 1: Is It Up? — stacker status

stacker status

Shows the deployment state, public IP, each service and its ports, and the emergency SSH line:

✓ Deployment #849 — status: completed
  IP:   178.105.97.168
── Services ──
  • app  (ports: 8000:8000)
  • ntfy (ports: 8080:80)

Two flags make it a daily driver:

• stacker status --watch — live refresh, with a terminal notification when the deployment reaches a terminal state.
• stacker status --json — machine-readable output for scripts (see the automation section).

Step 2: Live Container Health — stacker agent health

This is the remote equivalent of docker stats — per-container state, CPU, and memory, without SSHing in:

stacker agent health

Overall: ▶ running

CONTAINER          STATE        CPU%   MEM%   IMAGE
project-app-1      ▶ running    0.2    2.8    caronc/apprise:latest
project-ntfy-1     ▶ running    0.0    0.3    binwiederhier/ntfy:latest
statuspanel        ▶ running    0.0    0.1    trydirect/status:latest

Related views:

• stacker agent status — agent + container status for the deployment
• stacker agent containers — every container on the server
• stacker agent apps — apps deployed for the target
• stacker agent history — an audit trail of agent commands

Step 3: Logs, From the Terminal

Two ways to read logs — the local-friendly logs and the agent-fetched agent logs:

stacker logs -f                     # follow/stream
stacker logs --tail 200             # last 200 lines
stacker logs --since 2h             # last two hours
stacker logs --service ntfy         # one service only

stacker agent logs app --limit 200  # pull N lines via the agent

Step 4: Why Did It Pause or Fail? — stacker deployment events

When a deploy pauses, the generic status line rarely tells the whole story. The structured event log does:

stacker deployment events      # role-level, timestamped events
stacker deployment state       # canonical deployment state

This is where you find the real cause — a port already in use, a cloud quota (resource_limit_exceeded), an Ansible role that failed — rather than a wrapped "internal error." Make this your first stop for a stuck deployment.

Step 5: Act On It — restart, deploy-app

stacker agent restart <container>      # bounce a wedged container
stacker agent deploy-app --app <name> --image <img> --tag <tag>
stacker agent exec ...                 # raw agent command (advanced)

Config-Driven Health: the monitoring: Block

Turn on the agent and healthchecks declaratively in stacker.yml:

monitoring:
  status_panel: true
  healthcheck:
    endpoint: /health
    interval: 30s
  metrics:
    telegraf: true

# and per-service container healthchecks:
services:
  - name: api
    image: myorg/api:latest
    healthcheck:
      test: "CMD curl -f http://localhost:8000/health || exit 1"
      interval: 30s
      timeout: 5s
      retries: 3

These feed the state you see in status and agent health — the CLI just reports what the healthchecks decide.

The Differentiator: AI Analysis From the CLI

Because every command returns text, you can pipe it straight into an AI for a diagnosis. No copy-paste into a chat window:

stacker agent logs app --limit 200 | stacker ai ask "why is this container crashing?"
# or attach a file:
stacker ai ask "diagnose this" --context ./logs.txt

Pick your provider in stacker.yml — and note the privacy angle for a monitoring workflow:

ai:
  enabled: true
  provider: ollama          # local model — your logs never leave the server
  model: qwen2.5-coder
  endpoint: http://localhost:11434
  timeout: 0
  tasks: [troubleshoot, security]

Anthropic and OpenAI work too (provider: anthropic|openai with api_key: ${VAR}), but Ollama keeps every log line on your own box — exactly what you want when the "data" is your production logs. You can also run stacker ai --write to let a tool-capable model propose a fix to stacker.yml or .stacker/.

A Real Example: a Crashlooping Container

Here's the workflow end-to-end on an app that wouldn't stay up. First, health flags it:

stacker agent health
# project-app-1   ⏸ Restarting (1)   ghcr.io/dagucloud/dagu:latest

Then the logs show the smoking gun:

stacker agent logs app --limit 40
# Error: failed to create Wiki store: mkdir /var/lib/dagu/dags/wiki: permission denied
# ERROR Failed to create example DAG … open /var/lib/dagu/dags/…: permission denied

Hand it to the AI:

stacker agent logs app --limit 40 | stacker ai ask "why does this container keep restarting?"

…and the diagnosis is immediate: the named volume is created root-owned, but the image runs as a non-root user, so it can't write its data directory. The fix is a volume-ownership tweak (user: or an init that chowns the path) — not an app bug. You found it, diagnosed it, and know the fix, all from the terminal.

Living With It: Scripting & Alerts

The --json outputs turn monitoring into one-liners:

# alert if any container isn't running
# (agent health --json is an array of {name, status, cpu_pct, mem_pct, …})
stacker agent health --json 2>/dev/null \
  | jq -e 'all(.[]; .status == "running")' >/dev/null \
  || echo "⚠️  a container is down"

# handy aliases
alias sst='stacker status'
alias sh='stacker agent health'
alias slog='stacker logs -f --tail 100'

# cron: check every 5 minutes, notify on trouble
*/5 * * * * stacker status --json | jq -e '.status=="completed"' >/dev/null \
            || curl -d "deploy unhealthy" ntfy.example.com/alerts

Pair it with a healthcheck gate in CI (stacker ci), and route failures to Slack or ntfy — you can even wire that last hop with a Stacker PIPE.

Alert When a Container Goes Down

New in Stacker 0.3.2: a built-in stacker monitor command does exactly this. Add a monitoring.alerts block to stacker.yml and it watches container health, firing an alert once when a container stops and once when everything recovers (edge-triggered — no repeat spam):

monitoring:
  status_panel: true
  alerts:
    interval: 30
    on_recovery: true
    target:
      terminal: true                                   # terminal + desktop notification
      # url: "https://ntfy.example.com/alerts"         # or an HTTP webhook
      # method: POST

stacker monitor           # loop every `interval` seconds
stacker monitor --once    # single check — ideal for cron

# stop a container → the next check alerts:
# ● ⚠️ container problem: 1 not running (project-ntfy-1)
# 🔔 Stacker container alert: ⚠️ container problem: 1 not running (project-ntfy-1)

Alert state is persisted to .stacker/monitor.state, so one-shot --once runs stay edge-triggered across cron invocations. Targets can be a terminal/desktop notification, an HTTP webhook (ntfy, Slack), or (soon) a Stacker pipe.

The manual recipe (still handy, and how it works under the hood)

Before the built-in command — or if you want full control — you can build the same thing from agent health --json. A plain cron check fires every run while something is down, so make it edge-triggered to notify once on drop and once on recovery:

#!/usr/bin/env bash
# container-watch.sh — notify on state CHANGE only (no spam)
STATE=/tmp/stacker-health.state

if stacker agent health --json 2>/dev/null \
     | jq -e 'all(.[]; .status == "running")' >/dev/null; then
  now=up
else
  now=down
fi

prev=$(cat "$STATE" 2>/dev/null)
if [ "$now" != "$prev" ]; then
  [ "$now" = down ] && msg="⚠️ a container is DOWN" || msg="✅ all containers recovered"
  curl -s -d "$msg" https://ntfy.example.com/alerts     # or Slack / email
  echo "$now" > "$STATE"
fi

# check every 2 minutes
*/2 * * * * /usr/local/bin/container-watch.sh

Swap the curl for your channel of choice — Slack webhook, an SMTP one-liner, or route it through a Stacker PIPE to ntfy. Want per-container granularity? Change the jq to emit the offenders:

stacker agent health --json 2>/dev/null \
  | jq -r '.[] | select(.status != "running") | .name'
# → project-app-1

Troubleshooting Playbook

Frequently Asked Questions

Do I need to install anything on the server?

Set monitoring.status_panel: true and Stacker installs the agent during deploy. On an existing server, stacker agent install adds it.

Does agent health require SSH each time?

No — it talks to the on-box agent over the message bus. SSH is only the emergency fallback (stacker status prints the exact line).

Will sending logs to AI leak my data?

Only if you choose a hosted provider. With provider: ollama the model runs locally and your logs never leave the server.

Can Stacker notify me automatically when a container goes down?

Yes, as of Stacker 0.3.2 — add a monitoring.alerts block and run stacker monitor (see above). It edge-triggers a terminal/desktop notification or webhook when a container stops and again when it recovers. On older versions, wire the same behavior with the agent health --json + cron script above.

Can I get JSON for my own dashboards/scripts?

Yes — --json is available on status, agent health, and more. Pipe into jq.

Key Takeaways

• status → agent health → logs → deployment events covers 90% of day-to-day monitoring — no web UI
• The Status Panel agent answers live queries over the bus; SSH is only a fallback
• Every command is text, so stacker ai ask can diagnose logs directly — locally with Ollama for full privacy
• --json | jq turns monitoring into alerts, aliases, and cron in a few lines
• When a deploy is stuck, stacker deployment events is where the real cause lives