Edge AI SDK.

The Trillboards Edge AI SDK is a composable, cross-platform TypeScript toolkit that runs at the OS level on Linux and Windows devices. It brings the same audience sensing, federated learning, and ad delivery capabilities as the Android agent to any hardware — from LED backpacks to roadside billboards to indoor kiosks.

What It Does

• Real-time face detection and audience measurement via ONNX Runtime
• Attention scoring, dwell time tracking, and demographic classification
• On-device federated learning with Ed25519 VAS attestation
• Chromium kiosk browser lifecycle management for ad playback
• Cloud inference integration with Gemini for scene analysis
• Automatic device health monitoring with multi-level recovery
• Offline signal buffering with SQLite and batch reconnect flush
• Cross-platform: identical behavior on Ubuntu, Debian, Windows 10/11
• Interactive setup wizard, system diagnostics, and live status dashboard via CLI
• MCP server with 7 tools for AI agent fleet management via Claude, Cursor, or any MCP client
• Docker deployment (CPU/CUDA/OpenVINO) and Kubernetes Helm chart for fleet orchestration
• 3x3 screen-zone gaze tracking with creative-zone heatmaps for attention optimization
• Body pose engagement scoring — facing angle, lean detection, approach/depart classification
• Privacy-first speech intelligence — ambient keyword extraction with transcript zeroing
• HTTP status endpoint with Prometheus metrics on every device (port 9090)

The SDK is split into 8 composable packages. Install all of them with the umbrella package, or pick only what you need.

Installation

# Install the full SDK (all packages)
npm install @trillboards/edge-sdk

# Or install only what you need
npm install @trillboards/edge-core @trillboards/edge-sensing

Full Agent (All Features)

import { EdgeAgent } from '@trillboards/edge-sdk';

const agent = new EdgeAgent({
  platform: 'linux',
  deviceToken: 'YOUR_DEVICE_TOKEN',
  screenId: 'screen-001',
  venueType: 'outdoor',
  camera: { enabled: true },
  audio: { enabled: true },
  kiosk: {
    enabled: true,
    url: 'https://screen.trillboards.com',
  },
});

agent.on('started', ({ fingerprint, tier }) => {
  console.log('Agent started:', fingerprint, 'Tier:', tier);
});

agent.on('log', ({ level, message }) => {
  console.log(`[${level}] ${message}`);
});

await agent.start();

Sensing Only (No Kiosk)

Use individual packages for headless deployments where you only need audience sensing without ad playback.

import { DeviceIdentity, SocketManager, SensingConfig, VASWeightConfig } from '@trillboards/edge-core';
import { AudienceSensingService } from '@trillboards/edge-sensing';

// Use edge-core directly for sensing-only deployments
const identity = new DeviceIdentity(platformAdapter);
const fingerprint = await identity.getFingerprint();

const socket = new SocketManager({
  url: 'https://api.trillboards.com',
  fingerprint,
  deviceToken: 'YOUR_TOKEN',
  sensingConfig: new SensingConfig(),
  vasWeightConfig: new VASWeightConfig(),
});

socket.connect();

const sensing = new AudienceSensingService({
  fingerprint,
  socketManager: socket,
  sensingConfig: new SensingConfig(),
  vasWeightConfig: new VASWeightConfig(),
  camera: cameraProvider,
  faceModelPath: './models/blazeface.onnx',
});

await sensing.initialize();
sensing.start();

The EdgeAgent accepts a configuration object. Only platform, deviceToken, and screenId are required.

The sensing pipeline runs entirely on-device. Camera frames are processed locally through ONNX Runtime — no video ever leaves the device.

• BlazeFace ONNX model for real-time face detection (~400KB)
• YAMNet ONNX model for ambient audio classification
• Attention scoring with configurable yaw, pitch, and eye weights
• Dwell time tracking with multi-face assignment
• Age, gender, and emotion classification
• VAS (Verified Audience Signal) computation per aggregation window
• Observation signal classification for venue context
• Edge quality telemetry for model performance monitoring

Pipeline Architecture

Camera Frame (V4L2 / DirectShow / RTSP)
  → BlazeFace ONNX (face detection)
  → Face Tracking (assignment, dwell time)
  → Attention Scoring (yaw, pitch, eye weights)
  → Age/Gender/Emotion Classification
  → VAS Computation (Verified Audience Signal)
  → Socket.io emission → Backend ingestion

Train models on-device without sending raw data to the cloud. Gradients are compressed and cryptographically signed before upload.

• Gradient accumulation buffer with Float32Array precision
• Top-k sparsification — keeps top 10%, 90% bandwidth reduction
• Ed25519 VAS attestation via Node.js native crypto
• Model version checking with automatic download and validation
• Multi-slice context: venue_type x daypart x geo x device_profile

import { FederatedTrainer } from '@trillboards/edge-federated';

const trainer = new FederatedTrainer(machineId, {
  persistenceDir: '/data/trillboards',
});

// Start training with gradient upload callback
trainer.start(async (payload) => {
  socketManager.emitEvent('gradientUpload', payload);
});

For capabilities that exceed on-device compute (full scene understanding, detailed demographics), the SDK integrates with Gemini cloud inference.

• Scene analysis via Gemini Vision — demographics, venue context, content moderation
• Contextual targeting synthesis per aggregation window
• Conditional calling — only triggers when active viewers detected
• Configurable cadence per device to optimize API costs

Platform-specific adapters handle camera, audio, identity, and service management. Install only the adapter for your target OS.

• @trillboards/edge-platform-linux— V4L2 camera, PulseAudio, /etc/machine-id fingerprinting, systemd service
• @trillboards/edge-platform-windows— DirectShow + RTSP camera, WASAPI audio, registry GUID fingerprinting, Windows Service

Linux Deployment

# Install Linux platform adapter
npm install @trillboards/edge-platform-linux

# Deploy as systemd service
sudo cp trillboards-agent.service /etc/systemd/system/
sudo systemctl enable trillboards-agent
sudo systemctl start trillboards-agent

Windows Deployment

# Install Windows platform adapter
npm install @trillboards/edge-platform-windows

# Deploy as Windows Service via node-windows
npx trillboards-agent install

The Edge SDK operates at the OS level — it's a Node.js process that manages hardware, ML inference, and the kiosk browser. Connect and Ads SDK run inside the browser.

The Edge SDK includes an MCP (Model Context Protocol) server with 7 tools for managing DOOH device fleets from Claude Desktop, Cursor, or any MCP-compatible AI assistant. Point your agent at a fleet of screens and manage deployments, diagnostics, and telemetry conversationally.

MCP Configuration

{
  "mcpServers": {
    "trillboards-edge": {
      "command": "npx",
      "args": ["-y", "@trillboards/edge-sdk", "mcp"],
      "env": {
        "TRILLBOARDS_API_KEY": "your_key",
        "FLEET_ENDPOINT": "https://api.trillboards.com"
      }
    }
  }
}

Available Tools

Need Help?

Contact our developer support at developers@trillboards.com or visit the Support page.