# Scythe Command — Standalone Android App Plan
## Status: Phase 9 — Fresh Android App (ATAK Plugin Retired)
### Decision
ATAK plugin retired due to: CotPortListActivity crash (pre-existing ATAK bug), complex MOBAC tile API, GDAL legacy stack, JNI hook maintenance burden.
**New approach**: Standalone `ScytheCommandApp` — thin Android WebView shell loading `command-ops-visualization.html` directly from the RF Scythe orchestrator. All 35K lines of Cesium + RF viz reused immediately.
## Current Architecture
“`
ScytheCommandApp (APK on Pixel 7 Pro)
└── WebView → http://192.168.1.185:5001/command-ops-visualization.html
├── All /api/* calls → same origin (orchestrator:5001)
├── SSE streams → same origin (no CORS issues)
├── Cesium 3D globe (already working)
├── Recon Entity markers (1185 entities)
├── RF Hypergraph visualization
└── ScytheBridge JS interface (GPS, settings, toast)
“`
## Architecture (fully operational)
“`
ATAK Plugin (APK on Pixel 7 Pro)
│
├── CoT MapGroup listeners → entity.spawn / entity.move / entity.remove
│
├── SSE entity callback → rf.detection (sensor, freq, bearing, emitter_id)
│ → entity.move (non-RF entities)
│
└── EventStreamer → HTTP POST every 5s (buffer 200 events)
target: 10.107.190.84:8080 ← avf_tap_fixed network
▼
Debian VM (crosvm, CID 2049, 8-core Tensor G2, 3.8GB RAM, 70GB disk)
└── scythe-analytics.service (systemd, enabled, auto-restart)
└── scythe_vm_server.py v2 (Flask :8080)
├── ScytheDuckStore + ParquetPipeline (persistent DuckDB)
├── EventHypergraph (incremental, background rebuild 30s)
├── SpaceTimeCube (0.01° geo × 5s time voxels)
│
├── POST /api/events/ingest
├── GET /api/events/query?sql=SELECT…
├── GET /api/events/stats
├── POST /api/events/flush → Parquet blocks
│
├── GET /api/hypergraph/summary
├── GET /api/hypergraph/swarms (MEMBER_OF_SWARM edges)
├── GET /api/hypergraph/rf_triangulation (TRIANGULATED_FROM edges)
├── GET /api/hypergraph/co_movement (CO_MOVED_WITH edges)
├── GET /api/hypergraph/entity/<id>
│
├── GET /api/spacetime/query?lat&lon&radius_m&t0&t1
├── GET /api/spacetime/stats
│
└── GET /api/swarms (unified: DuckDB density + hypergraph)
“`
### VM Access
– SSH: `ssh -i /tmp/avf_scythe_key -o ProxyCommand=’adb -s … shell nc 10.107.190.84 22′ droid@avf-vm`
– Port forward: `adb forward tcp:8181 tcp:8080` → `curl localhost:8181/health`
– VM: Debian 13 Trixie, Python 3.13.5, DuckDB 1.5.0, PyArrow 23.0.1
### Systemd service
– Unit: `/etc/systemd/system/scythe-analytics.service`
– `ExecStartPre`: frees port 8080 via `fuser -k` before each start
– `Restart=on-failure`, `RestartSec=5`
– `MemoryMax=1G`, `CPUQuota=400%`
– `enabled` + `active`, NRestarts=0 ✅
## What was built
### ATAK Plugin (ATAKScythePlugin/)
– 4-tab UI: CONNECT / RF INTEL / MISSIONS / SWARMS
– RF node map layer + animated swarm layer (pulsing rings, velocity arrows)
– OkHttp REST + SSE client connecting to rf_scythe_api_server.py
– 16KB ELF compliant: extractNativeLibs=false, useLegacyPackaging=false
– **EventStreamer.java** — buffers 200 events, flushes to /api/events/ingest every 5s
– **APK: app/build/outputs/apk/debug/app-debug.apk (6.1 MB)**
– APK installed on device ✅ — `android:exported=true` + `<queries>` block (Android 11+ visibility fix)
### Python Intelligence Modules
– scene_event_schema.py — 15-event canonical schema ✅
– scene_event_log.py — SQLite-WAL ledger, .atakrec export ✅
– scene_replay_engine.py — deterministic replay, scrub, fork ✅
– scene_event_compressor.py — 7.9× columnar compression, fixed dict-encode bug ✅
– scene_hypergraph.py — RF triangulation, swarm, co-movement edges ✅
– scene_spacetime_cube.py — 0.14ms radius+time query ✅
– cluster_swarm_engine.py — geo-bucket cluster detection → CoT ✅
– tak_swarm_emitter.py — PyTAK CoT emitter ✅
– **scene_duckdb_store.py** — DuckDB event store, 54ms bulk insert / 10ms scrub ✅
– **scene_parquet_pipeline.py** — ZSTD Parquet cold store, 3× compression, 18ms read ✅
### Server (rf_scythe_api_server.py)
– Swarm routes: /api/clusters/swarms, /stream, /cot
– Replay routes: /api/replay/session/start|end|snapshot|events|state|export
– **DuckDB routes:**
– POST /api/events/ingest — bulk Android→DuckDB ingestion
– GET|POST /api/events/query?sql=… — arbitrary SELECT analytics
– GET /api/events/stats — store statistics
– GET /api/events/export/parquet — stream Parquet file to client
– GET /api/events/blocks — list Parquet cold-storage inventory
– POST /api/events/flush — partition hot store into Parquet blocks
## Phase 7: Plugin Loaded ✅ — End-to-end pipeline validation
### Next steps
– [ ] Tap RF Scythe toolbar icon → verify 4-tab UI opens (CONNECT / RF INTEL / MISSIONS / SWARMS)
– [ ] Verify EventStreamer connects to VM 10.107.190.84:8080 (CONNECT tab shows green)
– [ ] Generate RF detection events → confirm /api/events/stats count increases
– [ ] Test RF signal dot map layer renders on ATAK map
– [ ] Surface /api/hypergraph/rf_triangulation results back to RF INTEL tab
– [ ] Emit swarm.update events from SwarmLayer on cluster membership changes
– [ ] Surface /api/spacetime/query for operator “query area” map gesture
—
## Known Blocker (RESOLVED): ATAK CIV 4.6.0 on Android 16
– ATAK CIV crashes at startup on Pixel 7 Pro (Android 16 / SDK 36)
– Root cause: libsqlite3.so removed from Android 16 system image; commoncommo NDK
binaries linked against pre-bionic-hardening ABI; FORTIFY pthread_mutex crash
– **Our plugin installs and is visible** — AppsFilter BLOCKED resolved
– **Plugin cannot load because ATAK itself crashes before loading any plugins**
– Solutions:
1. Android emulator API 30-33 (immediate, no hardware needed)
2. Second physical device with Android 11-14
3. Recompile ATAK from source (atak/ATAK/) with bundled SQLite + NDK r26+
4. Wait for TAK.gov to release ATAK CIV 5.x with Android 15/16 support
## Next Steps
### Immediate (map stability)
– [x] Fix GLMapView.inverseImpl() NPE — null guard on `lastsm.displayModel`
– [x] Fix MapTouchController.onGestureStart() + onScale() NPE — same null guard pattern
– [ ] Confirm map tap no longer crashes (install test pending)
– [ ] Check if any other touch controllers have the same `sm.displayModel` access pattern
### Short-term (pipeline validation)
– [ ] Verify EventStreamer connects to VM (CONNECT tab shows green)
– [ ] Generate RF detection events → confirm /api/events/stats count increases
– [ ] Test RF signal dot map layer renders on ATAK map
– [ ] Surface /api/hypergraph/rf_triangulation results back to RF INTEL tab
– [ ] Emit swarm.update events from SwarmLayer on cluster membership changes
– [ ] Surface /api/spacetime/query for operator “query area” map gesture
### Medium-term strategic: Plugin-level Cesium renderer override
The native TAK engine has fundamental fragility (null displayModel NPEs, GDAL bloat,
no first-class 3D/WebXR path). The recommended path forward:
1. **Prove the override hook** — in ScytheMapComponent, grab the GLSurfaceView from
ATAKActivity and set a custom GLSurfaceView.Renderer that wraps Cesium Native renders
into the existing GL context. ATAK UI stays untouched; overlays migrate to Cesium primitives.
2. **RF volumes + swarm viz** — Cesium Native handles volumetric RF propagation domes,
sensor cones, animated swarm centroids/particles natively via 3D Tiles + Entity API.
3. **Long-term** — intercept EngineLibrary native lib load, drop libtakengine_cesium.so
stub (~20 critical JNI methods → Cesium). Full 3D/streaming, no GDAL, smaller APK.
## Phase 8 Strategy: Staged JNI Hook Rollout
**Why staged:** The JNI hooking approach is powerful but requires careful C++ implementation.
Instead of integrating Cesium immediately, we’re building the foundation layer-by-layer.
**Stage 1 (✅ COMPLETE):** RF Scythe plugin stable + HookManager integrated
**Stage 2 (NEXT):** JNI hook library (libscythe_hook.so) — dlsym interception
**Stage 3 (FUTURE):** Full Cesium Native renderer integration
### Phase 8.1 Complete ✅
– [x] RF Scythe plugin loaded and running (CONNECT/RF INTEL/MISSIONS/SWARMS tabs functional)
– [x] HookManager.java created with static initializer
– [x] ScytheLifecycle integrated to trigger HookManager.ensureLoaded() early
– [x] Plugin lifecycle logs confirm hook system attempting to load (graceful fail if .so missing)
– [x] No crashing when libscythe_hook.so unavailable — continues with ATAK rendering
– [x] Build: lint disabled (Instantiatable), assemble succeeds, APK installed ✅
1. **Basemap not rendering:** `DatasetDescriptorFactory2.create()` fails when generating MOBAC XML
– Legacy ATAK raster stack is fragile
– This is exactly why JNI hooking is the right approach
2. **Native build complexity:** Android NDK + CMake + Cesium dependencies require careful setup
– Created: `tak_engine_hook.cpp`, `cesium_renderer_wrapper.cpp`, `HookManager.java`
– Will integrate into build pipeline in Phase 8.2
### Immediate Fallback (Phase 8.1)
Keep RF Scythe plugin working without basemap for now:
– Plugin connects to VM ✅
– CONNECT tab shows “STREAMING” ✅
– RF INTEL, MISSIONS, SWARMS tabs functional ✅
– RF signal dots + swarm rings render (in 2D, no globe) ✅
– Map canvas is blank but not crashing ✅
### Path to 3D Globe (Phase 8.2-8.3)
Once JNI hook layer is complete:
– Every ATAK map call → redirects to Cesium
– 3D globe renders, streaming 3D Tiles work
– RF overlays composite on top
– No MOBAC/XML/tile-reader fragility
### JNI Hook Layer Architecture
**Interception chain:**
“`
Java: GLMapView.drawFrame()
↓ JNI call
JVM: dlsym(“Java_com_atakmap_map_opengl_GLMapView_drawFrame”)
↓ our hook intercepts
dlsym hook: is_critical_symbol() → true
↓ returns our implementation
Our handler: generic_handler → Java_com_atakmap_map_opengl_GLMapView_drawFrame_impl
↓ delegates to Cesium
Cesium: CesiumRenderer::drawFrame()
↓ renders 3D globe + overlays
OpenGL framebuffer
“`
**Load order critical:**
1. `HookManager.initialize()` calls `System.loadLibrary(“scythe_hook”)` early (static block)
2. This must happen **before** ATAK’s ATAKActivity loads `libtakengine.so`
3. Once libscythe_hook.so is loaded, our dlsym hook is active
4. When ATAK tries to load libtakengine.so and resolve its JNI symbols, we intercept
### Symbol list (~20 critical):
“`
Globe lifecycle: create, destroy, resize, setDisplayMode
Rendering: drawFrame (core)
Projection: forward, inverse, getProjection
Layer rendering: DatasetRasterLayer2_drawImpl, TileClient_getTile
Coordinate xform: CoordsUtility_forward/inverse, WebMercatorProjection_*
“`
Each symbol → routes to Cesium Native implementation in `cesium_renderer_wrapper.cpp`.
## ATAK Build Notes
– Source: AndroidTacticalAssaultKit-CIV-main/atak/ATAK/
– Build: `JAVA_HOME=/opt/amazon-corretto-11.0.30.7.1-linux-x64 ./gradlew assembleCivSmallDebug -PskipNativeBuild=true`
(Gradle 6.9.1 requires Java 11 — Java 21 causes Groovy ASM class version 65 error)
– local.properties requires absolute paths for sdk.dir and takDebugKeyFile
– NOP patch at 0xbfe280 in libtakengine.so — survives rebuild (pre-compiled .so in jniLibs)
– Patched files (source patches, applied once):
– GLMapView.java: null guard for lastsm.displayModel in inverseImpl() (line 909)
– MapTouchController.java: null guards in onGestureStart() + onScale()
## Build fixes applied (for reference)
– AGP 8.3.2 + Gradle 8.6 + Java 21
– Manifest: android:exported=true on component activity; <queries> block for ATAK CIV
– ic_scythe.xml: replaced <circle> with path (AAPT2 compatibility)
– GLMapView.forward(GeoPoint,float[]) → forward(GeoPoint) returns PointF
– GLMapView.getBounds()/getHeight() → northBound/southBound/_top/_bottom fields
– AbstractLayer.dispatchOnLayerVisible() → dispatchOnVisibleChangedNoSync()
– PluginTool → AbstractPluginTool (constructor takes Drawable, no override needed)
– SseStreamClient: InterruptedException not thrown by connect() → catch Exception