What This Covers
A technical walkthrough of getting Stacker's PIPE system working end-to-end: creating pipes, activating them on remote agents, and triggering execution between containers on a live server. We hit every wall possible - Docker networking, agent modes, deployment hashes - and documented the fixes.
Quick Answer
# The working PIPE flow (4 commands):
stacker pipe create directus chatwoot --source-endpoint "POST /items" --name "directus-chatwoot"
stacker pipe activate --server <IP> --pipe <PIPE_ID>
stacker pipe scan directus --server <IP>
stacker pipe trigger directus --server <IP> --pipe "directus-chatwoot"The Stack
- Directus - headless CMS, source of data
- Chatwoot - customer support platform, target for data
- Stacker - deployment platform orchestrating both
The promise: a single stacker pipe trigger command fetches data from Directus and pushes it to Chatwoot. In practice, getting there took four days of debugging across three servers, two Docker networks, and an orphaned AMQP consumer.
Day 1: Creating the Pipe
The stacker pipe create command needed a --name flag. Previously, it always prompted interactively, which broke CI and scripting workflows.
stacker pipe create directus chatwoot \
--source-endpoint "POST /items" \
--target-endpoint "POST /api/v1/conversations" \
--source-fields "name,email,message" \
--target-fields "content" \
--name "directus-chatwoot"This created the pipe template locally. Remote activation worked on the first try - the pipe instance was registered with the Stacker API and linked to the deployment.
Takeaway: The pipe creation flow (CLI → API → database) is solid. The bottleneck was always the execution side.
Day 2: The Agent Mystery
With the pipe created, we tried stacker pipe trigger. It hung for 30 seconds, then timed out. No error, no output. The command was reaching the server, but nothing was executing.
Daemon Mode vs. Web Server
The Dockerfile's default entrypoint starts a web server. But for pipe execution, the agent needs to run as a daemon - a long-polling background process that pulls commands from the Stacker API and executes them locally.
# The default entrypoint (web server mode):
ENTRYPOINT ["/usr/local/bin/status", "serve", "--with-ui"]
# What we needed (daemon mode):
docker run -d \
--name statuspanel_agent \
--entrypoint /usr/local/bin/status \
-c /app/config.json \
...The Deployment Hash Problem
Even with the correct mode, the agent returned 403 errors. Every deployment in Stacker gets a unique hash (deployment_<hex>), and the agent authenticates using this hash. After redeploying, the old hash was invalid.
# Find current deployment hash
curl -s "https://stacker.try.direct/api/v1/project/330/deployments" \
-H "Authorization: Bearer $TOKEN" | jq '.[0].deployment_hash'
# → "deployment_abc12345-6789-0def-ghij-klmnopqrstuv"Lesson: Every time you redeploy, check if the agent's deployment hash is still valid.
Day 3: Network Isolation
With the agent registered and polling, pipe scan worked - it found the Directus container. But pipe trigger still timed out.
Two Networks, One Problem
The target server had two Docker networks:
project_app-network- Bridge network for project containers (Directus, Chatwoot)default_network- External network for the status panel agent
Docker bridge networks are isolated by default. Containers on different networks cannot reach each other by IP. The agent could resolve container names but couldn't actually connect to them.
# Fix: connect the agent to the project network
docker network connect project_app-network statuspanel_agentDay 4: The Execution
With networking and naming fixed, pipe trigger finally executed. The agent received the command, parsed it, and attempted to run:
failed to fetch trigger_pipe source: source container request failed with code 127:
/bin/sh: curl: not found
The Directus container doesn't have curl installed. The agent tries to fetch source data by running curl inside the target container via docker exec.
The important thing: the mechanism works. The command was received, executed, and reported back. The failure is at the application level (missing curl), not the infrastructure level.
What We Discovered: Two Parallel Architectures
While debugging, we found agent-executor - a standalone Rust binary that connects to RabbitMQ and listens for StepCommand messages. We built it, deployed it, and watched it crash repeatedly. Here's the thing: no code in Stacker ever publishes StepCommand to RabbitMQ.
Stacker has two execution paths that were never connected:
- HTTP long-polling (statuspanel_agent) - Working. Used by
stacker pipe triggerandpipe scan - AMQP (agent-executor) - Orphaned, never wired up. The infrastructure exists but no publisher was ever implemented
The Complete Working Flow
- Create pipe (CLI) - Creates pipe template and instance in Stacker database
- Activate pipe (CLI) - Enqueues activation command to agent
- Scan endpoints (CLI) - Agent runs probe on target container, returns available endpoints
- Trigger execution (CLI) - Agent receives command via long-poll, executes inside source container, pushes data to target, reports result back
Lessons Learned
1. Docker Networking Is the #1 Gotcha
Every cross-container communication issue was network-related. The agent must be on the same Docker network as the containers it manages. This is not obvious from the documentation.
2. Deployment Hashes Are Ephemeral
Every deploy creates a new hash. If the agent is registered with an old hash, all commands fail with 403. After any deploy, re-register the agent.
3. Daemon Mode Is Not the Default
The Dockerfile entrypoint starts a web server. For pipe execution, you need the daemon. This is a configuration detail that's easy to miss.
4. AMQP Was Designed but Never Connected
The agent-executor binary and RabbitMQ integration exist in the codebase, but no code publishes to the pipe_execution exchange. The HTTP long-polling path is the only one that works.
Frequently Asked Questions
Why does stacker pipe trigger hang and timeout?
The most common cause is Docker network isolation. The agent must be on the same bridge network as the project containers. Run docker network connect project_app-network statuspanel_agent to fix it.
What is the difference between daemon mode and web server mode?
Web server mode (serve --with-ui) hosts the status panel dashboard. Daemon mode polls the Stacker API for commands and executes them locally. Pipes require daemon mode.
Do I need to re-register the agent after every deploy?
Yes. Each deployment creates a new hash. The agent authenticates with the hash it was registered with. If the hash changes, the agent returns 403.
Why doesn't curl work inside containers?
Many minimal Docker images (like Directus) don't include curl. The pipe executor currently assumes curl is available. A future update will use an alternative HTTP client.
What's Next
- Install curl in target containers or modify the pipe executor to use a different HTTP client
- Wire up the AMQP path for event-driven pipe execution
- Auto-register agents with current deployment hashes after redeploy
Key Takeaways
- Stacker's PIPE architecture is sound - the gaps are in networking and configuration
- Docker network isolation is the most common blocker for cross-container pipes
- Agent must run in daemon mode, not web server mode
- Deployment hashes are ephemeral - re-register after every deploy
- The HTTP long-polling path works end-to-end; AMQP is unused infrastructure