Trillboards Sensing SDK Real-Time Audience Intelligence

The Sensing SDK ships the canonical Trillboards sensing surface as a single Maven artifact for Android. It runs the face, audio, speech, BLE, WiFi and discovery pipelines on the device and sends Trillboards a record for each detected person (estimated age range, gender, dominant emotion, dwell time and time spent looking at the screen), speech transcripts, the Bluetooth, WiFi and local-network identifiers nearby devices broadcast, the screen's GPS position, and camera frames selected for cloud analysis. On the full sensing profile it also computes a face-identity template, a vector of 512 numbers that Trillboards uses to recognise the same person across screens and days. No audio recording leaves the device. The agent transmits the device's resettable Google Advertising ID (GAID) when available, used for cross-device frequency capping and partner attribution; the agent honors LIMIT_AD_TRACKING.

What ships in production today

• On-device demographics, FaceXFormer (10-year age bins, gender estimates)
• Audio classification, MediaPipe AudioClassifier (YAMNet event taxonomy)
• Speech recognition, Moonshine ASR via sherpa-onnx
• Audio diarization, multimodal clip prompt schema
• BLE co-viewing, beacon scan, GATT 0x180A enumeration, manufacturer parsers (iBeacon, Eddystone, Continuity, FastPair, Tile)
• WiFi environment, RSSI variance, channel congestion, AP density
• Multi-protocol device discovery, mDNS, SSDP, ARP, HTTP probes
• Vertex multimodal embeddings (1408-dim per observation)

Android is the primary platform. The full sensing surface ships as a single Maven AAR, one Gradle dependency line, no transitive setup.

1. Register the Maven registry

Add the registry block to your root settings.gradle.kts so Gradle can resolve com.trillboards:agent-core. maven.trillboards.com serves anonymously, no auth tokens needed.

dependencyResolutionManagement {
    repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
    repositories {
        google()
        mavenCentral()
        maven { url = uri("https://maven.trillboards.com") }
    }
}

2. Add the dependency

dependencies {
    implementation("com.trillboards:agent-core:1.22.0")
}

Source repo (Apache 2.0): https://github.com/trillboards/agent-core. maven.trillboards.com publishes each release, and this page resolves the version straight from the registry. 1.22.0 is the current release. Pin it exactly, or use 1.22.+ to float the minor line.

3. Initialize the SDK

import com.trillboards.sdk.TrillboardsSensingSdk

class MyApplication : Application() {
    override fun onCreate() {
        super.onCreate()
        TrillboardsSensingSdk.start(
            context = applicationContext,
            partnerApiKey = BuildConfig.TRILLBOARDS_PARTNER_API_KEY
        )
    }
}

4. Declare manifest permissions

As of SDK 1.12 the discovery permission set (BLE, WiFi, location, multicast, network) merges into your app automatically from the SDK's library manifest, you only declare the two sensors that must stay host-owned: CAMERA and RECORD_AUDIO. The agent only activates the sensors your venue actually exposes; every hardware feature is declared optional so an APK without cameras, BLE, WiFi, or GPS still installs cleanly and the SDK degrades per-source (reporting a machine-readable skip_reason instead of silently going dark).

<uses-feature android:name="android.hardware.camera" android:required="false" />
<uses-feature android:name="android.hardware.microphone" android:required="false" />

<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />

The full set the SDK's manifest contributes (for review / allowlisting, do not re-declare unless noted):

<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
<uses-permission android:name="android.permission.CHANGE_WIFI_STATE" />
<uses-permission android:name="android.permission.CHANGE_WIFI_MULTICAST_STATE" />
<uses-permission android:name="android.permission.NEARBY_WIFI_DEVICES"
    android:usesPermissionFlags="neverForLocation" />
<uses-permission android:name="android.permission.BLUETOOTH_SCAN"
    android:usesPermissionFlags="neverForLocation" />
<uses-permission android:name="android.permission.BLUETOOTH_CONNECT" />
<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />

Location note: GPS reporting is on by default and is part of the data contract your venues sign up for, grant ACCESS_FINE_LOCATION at runtime (MDM-managed fleets pre-grant it). If another dependency in your app declares ACCESS_FINE_LOCATION with maxSdkVersion="30" (many BLE/location plugins do), the SDK's manifest already carries a tools:node="replace" guard; if the capped declaration lives in YOUR app manifest, add the same marker there or GPS silently stops at Android 12+.

For non-Android partners, Linux digital-signage players, Windows kiosks, embedded edge boxes, install the Node-based @trillboards/edge-sdk. It exposes the same sensing event surface backed by ONNX runtimes and BlueZ.

npm install @trillboards/edge-sdk

import { EdgeAgent } from '@trillboards/edge-sdk';

const agent = new EdgeAgent({
  platform: 'linux',
  deviceToken: 'YOUR_DEVICE_TOKEN',
  screenId: 'screen-001',
  camera: { enabled: true },
  audio: { enabled: true },
});

await agent.start();

See the Edge SDK reference for federated learning, kiosk management, and the MCP fleet-control surface.

Creatives running inside a partner WebView consume sensing data through the bridge contract exposed by @trillboards/ctv-measurement. No camera permissions, no on-device models, the host agent fans out measurement snapshots and your WebView reads them.

npm install @trillboards/ctv-measurement

// Inside a WebView creative running on a Trillboards-hosted screen
const bridge = (window as any).__TRILL_BRIDGE_V1__;

if (bridge?.getMeasurementSnapshot) {
  const snapshot = await bridge.getMeasurementSnapshot();
  console.log('attention', snapshot.attention);
  console.log('audience', snapshot.audience);
  console.log('discovery', snapshot.discovery);
}

See the CTV Measurement reference for the full bridge schema, GIVT-deduped impression ingestion, and QR attribution.

Every knob below is partner-tunable at construction time or via server-pushed sensing-config updates. Defaults are tuned for retail tablets at typical mounting distances; only override when your venue calls for it.

From SDK 1.12 the heartbeat envelope (POST /openrtb/v1/heartbeat, ~65s effective cadence: a ~30s BLE scan window plus the 30s interval) carries the device-truth fields below alongside the audience and device-presence signals. You do not need to send any of this yourself; the SDK assembles it.

skip_reason semantics

When a collector cannot run, it reports a machine-readable reason instead of silently emitting zero rows, so a missing runtime grant looks different from an empty room. Counters accumulate per source between heartbeats and reset on send. Trillboards monitors fleet-level permission_denied and scanner_unavailable rates, which is how a mis-provisioned rollout gets caught quickly.

What the Sensing SDK sends, how long Trillboards keeps it and who receives it.

• Per-person records, one for each person the camera detects: estimated age range, gender and dominant emotion, dwell time and time spent looking at the screen, with head orientation, facial measurements and upper-body points for each analysed frame. Trillboards keeps them for 400 days, and its encrypted archive keeps the records with no scheduled deletion.
• Face-identity templates, on the full sensing profile: a vector of 512 numbers for each detected person, which Trillboards clusters into person identifiers that recognise the same person across screens and days and links to the device keys of the phones seen with them. Templates are kept for 400 days and are never sold or shared with a data partner.
• Camera frames, selected frames go to Trillboards and Google Vertex AI, where a cloud model estimates attributes including ethnicity, age band, social composition, purchase intent, attire and lifestyle segments. Trillboards keeps the frames for 400 days, and the frames captured between 5 May and 21 June 2026 are kept as a model-evaluation set with no scheduled deletion. No video is recorded.
• Speech, the agent transcribes speech on the device. Trillboards stores the transcript and a cloud model reads it for purchase intent, brands mentioned and sentiment. No audio recording is stored or transmitted.
• Radio identifiers, the Bluetooth, WiFi and local-network identifiers nearby devices broadcast, stored as received and as device keys; Trillboards links a device key to the person seen with the device and to mobile advertising IDs.
• Advertising ID, the agent transmits the device's Google Advertising ID (GAID) when available, used for cross-device frequency capping and partner attribution. The agent honors LIMIT_AD_TRACKING.
• Defaults, sensing, face detection, audio classification, speech intelligence, demographics and advertising-ID collection are on by default, each with a SensingSdkConfig flag that turns it off. GPS capture runs whenever the app holds the location permission.
• Data partners, Trillboards sells and shares audience measurement and location data with its data partners, including 375ai and Intuizi. The Data Processing Addendum at trillboards.com/legal/data-processing sets out every category, how long each is kept and who receives it.
• Transparent disclosure, partners are responsible for venue-side signage. The SDK exposes a status surface so your kiosk shell can render the active sensing posture to viewers on request.

Capabilities differ across the three platform SDKs because the underlying runtimes do. The table below is honest about per-platform gaps so you can pick the right surface for your hardware.

Need help?

Email engineering@trillboards.com for integration support. For native IMA SDK consumers, the partner-native-ima-sdk guide's Section 12 covers the Maven coordinate, GitHub Packages registry block, and minimal init snippet from a partner-ad-serving perspective.