Integration & Testing

Always keep your eyes on integration. Design for testability from day one.

Always Think Integration

Keep your eyes on the prize: Every subsystem you design will eventually talk to something else. If you wait until the end to think about integration, you'll discover incompatible interfaces, conflicting assumptions, and impossible timelines. Design with integration in mind from day one. Ask "how will this connect?" before you ask "how will this work?"

Integration is where theory meets reality. Your beautifully designed subsystems must physically connect, electrically communicate, and logically cooperate. Most projects fail not because individual components are bad, but because they don't work together.

The key to successful integration: incremental assembly, relentless testing, and obsessive documentation of interfaces.

Design for Testability: Observability & Test Points

Assume things will break: They always do. The question is whether you can diagnose and fix them quickly. Test points and observability aren't optional luxuries—they're essential infrastructure. Design them in from the start, because retrofitting debug access is painful and expensive.

Where to Add Test Points

Power Rails

Every major voltage rail needs measurement access:

  • Voltage test points (before and after regulators)
  • Current sense resistors or Hall sensors
  • Load switches to isolate subsystems
  • Indicator LEDs for quick visual checks

Data Buses

Communication interfaces need probe access:

  • SPI/I2C/UART: expose CLK, MOSI, MISO, CS lines
  • CAN bus: high-side and low-side differential pairs
  • Ethernet: TX/RX pairs accessible for scope probing
  • Logic analyzer headers for multi-signal capture

RF Chains

Radio paths need multiple measurement points:

  • RF test points after PA, before antenna
  • Directional couplers for TX power monitoring
  • RSSI or AGC voltage monitoring for RX
  • LNA bypass for troubleshooting RX issues

Debug Interfaces

Every processor needs debug access:

  • JTAG or SWD exposed (don't bury it)
  • UART console for printf debugging
  • Dedicated debug connector (not shared pins)
  • Boot mode selection (normal vs debug/recovery)

What to Measure

Subsystem Key Measurements Why It Matters
Power Voltage, current, ripple, efficiency Detect brownouts, shorts, thermal issues
Communications Signal integrity, bit error rate, latency Verify protocol timing, catch data corruption
RF TX power, RX sensitivity, SNR, frequency error Ensure link budget margins, diagnose range issues
Timing Clock frequency, jitter, phase noise Real-time systems fail with timing violations
Thermal Component temperatures, gradient mapping Prevent thermal runaway, validate cooling
Built-in diagnostics: Don't just add test points—add self-test routines, health checks, and telemetry hooks. Software should report its own status: "watchdog fired 3 times," "I2C bus timeout on sensor 2," "temperature approaching limit." Make the system tell you what's wrong.

Lock Interfaces First, Internals Second

Interfaces are contracts: Once defined, they're expensive to change. Lock down interfaces early so teams can work in parallel. Internal implementations can evolve, but interface changes propagate across the entire system. Invest heavily in Interface Control Documents (ICDs) upfront.

What to Define in ICDs

Electrical Interfaces

  • Voltage levels (3.3V, 5V, differential)
  • Current draw (max, average, inrush)
  • Connector type and pinout
  • Impedance matching (for high-speed)
  • Grounding and shielding strategy
  • ESD protection requirements

Mechanical Interfaces

  • Mounting hole patterns and spacing
  • Envelope constraints (max dimensions)
  • Connector orientation and access
  • Cable routing and bend radius
  • Thermal interface (heatsink contact)
  • Keep-out zones for other subsystems

Data Interfaces

  • Protocol (UART, SPI, CAN, Ethernet, USB)
  • Baud rate or clock frequency
  • Message formats (packet structure)
  • Timing requirements (setup, hold, latency)
  • Error handling (CRC, retry, timeout)
  • Flow control mechanism

Power Sequencing

  • Startup order (which rails first)
  • Delay requirements between rails
  • Brownout behavior (what happens below threshold)
  • Shutdown sequence (graceful vs emergency)
  • Hot-swap capability (if required)
  • Fault isolation (prevent cascade failures)
Version Control Your ICDs

ICDs are living documents. Track every change, get sign-off from affected teams, and maintain revision history. A single "quick fix" to an interface without updating the ICD causes integration disasters. Treat ICD changes like code changes: review, approve, document.

Incremental Integration Strategy

Don't integrate everything at once: Big-bang integration is a recipe for chaos. You end up with a pile of broken parts and no idea where to start debugging. Build incrementally: power-on first, then add one interface at a time, verify at each step.

Integration Steps

  1. Power-On Test: Before anything else, verify power rails come up cleanly. Measure voltages, check for shorts, verify sequencing. No point testing data if power is broken.
  2. One Interface at a Time: Add subsystems sequentially. Computer boots → add sensor → verify I2C communication → add actuator → verify SPI. Isolate failures to specific interfaces.
  3. Known-Good Baselines: After each successful integration step, save that configuration. If the next step breaks, you can roll back to the last working state. Version control for hardware integration.
  4. Stub Out Subsystems: Don't wait for final hardware. Use simulators, dev boards, or dummy loads to test interfaces early. Replace stubs with real hardware incrementally.
  5. Integration Checkpoints: Define clear pass/fail criteria at each step. Document what works, what doesn't, and what workarounds were needed. Lessons learned feed forward.
The 80/20 rule of integration: 80% of integration time is spent debugging interfaces. Only 20% is actual assembly. Plan accordingly. Schedule integration time as if you're debugging, not building.

Building Without Hardware in Hand

Reality check: You'll rarely have final hardware when development starts. Deadlines are fixed, but hardware delivery slips. Smart engineers derisk by building incrementally, testing on approximations, and designing for testability. Don't wait for the perfect environment—create strategic stepping stones.

Development Timeline (Without Final Hardware)

1

Weeks 1-4: Simulations & Models

Build software models before touching hardware. Mathematical models for RF link budgets, thermal simulations, power analysis. Software-in-the-loop (SIL) for algorithm development. Remove algorithmic uncertainty before hardware complexity.

Deliverable: Proven algorithms, validated assumptions, identified risks

2

Weeks 5-10: Development Boards / COTS

Use off-the-shelf hardware to validate interfaces early. Arduino, Raspberry Pi, STM32 dev boards—whatever's close enough. Prove software runs on real hardware with interrupts and timing constraints. Test interface protocols (SPI, I2C, UART) with actual devices.

Identify gotchas: Race conditions, buffer overflows, timing violations, real-world noise

3

Weeks 11-16: Breadboard / Proto PCBs

Build functional approximation with target components. Use actual chips, connectors, power supplies you'll fly. Electrical verification: signal integrity, noise, power consumption. Integration testing between subsystems. Iterate quickly—breadboards are disposable.

Find problems now: Before committing to expensive PCB fabrication

4

Weeks 17-24: Engineering Model

First real PCB, actual form factor, representative hardware. Not flight-ready, but electrically and mechanically similar. Full system integration: all subsystems talking. Environmental testing if available (thermal, vibration).

Purpose: Find design flaws before flight hardware commits

5

Week 25+: Flight Hardware

Final hardware arrives—but your software already works. Minimal surprises because you've tested approximations. Focus on qualification testing, not basic functionality. Timeline risk reduced: integration is incremental, not big-bang.

Result: Confidence from proven performance on similar hardware

Key insight: Each stage removes one type of uncertainty. Simulation removes algorithmic uncertainty. Dev boards remove software/hardware interface uncertainty. Breadboards remove electrical uncertainty. Engineering model removes mechanical/thermal uncertainty. By the time you get flight hardware, you've validated everything except the exact components.

Technology Choices: Popular vs Appropriate

Don't follow trends blindly: Just because everyone uses Kafka doesn't mean your system needs it. Analyze your actual requirements. Sometimes the "boring" technology is exactly right. Sometimes you need something niche. The question is always: does this solve my problem better than alternatives?

Common Technology Trade-Offs

Message Queues: Kafka vs Alternatives

Kafka is great when:

  • High throughput (millions of messages/sec)
  • Distributed system with multiple consumers
  • Need message replay and persistence
  • Have ops team to manage cluster

Simpler alternatives when:

  • RabbitMQ: Easier ops, good routing, lower throughput OK
  • Redis Streams: In-memory speed, simpler setup
  • NATS: Lightweight, low latency, embedded use cases
  • Direct socket: Ultimate simplicity for point-to-point

Databases: SQL vs NoSQL vs File

PostgreSQL when:

  • Need transactions and consistency
  • Complex queries and joins
  • Data has clear schema

Alternatives:

  • SQLite: Single node, embedded, zero-config
  • MongoDB: Schema flexibility, document-oriented
  • InfluxDB: Time-series data (telemetry, logs)
  • Flat files: Telemetry dumps, log rotation, simplicity

Networking: REST vs gRPC vs Custom

REST when:

  • Human-readable debugging matters
  • Widely supported clients
  • Request/response pattern sufficient

Alternatives:

  • gRPC: Binary efficiency, streaming, type safety
  • MQTT: Pub/sub, low bandwidth (IoT, embedded)
  • WebSockets: Bidirectional, real-time updates
  • Raw TCP/UDP: Ultimate control and efficiency

Processing: Microservices vs Monolith

Microservices when:

  • Large teams, independent deployments
  • Different scaling needs per service
  • Polyglot requirements (multiple languages)

Monolith when:

  • Small team (< 10 people)
  • Simple deployment preferred
  • Network latency matters
  • Starting new project (defer complexity)
Decision framework: (1) What problem am I solving? (2) What's the simplest solution? (3) Does complexity pay for itself? (4) Can I test/maintain it? Popular technology attracts talent and has good docs, but don't let that override fit-for-purpose analysis.

Documentation as Integration Tool

If it's not documented, it doesn't exist: Integration depends on shared understanding. When subsystems are developed by different teams (or even different people), documentation is the only way to ensure everyone's building compatible pieces. Skimp here and pay during integration.

Essential Integration Documents

Update Documentation in Real-Time

Don't wait until after integration to document what you learned. Capture it immediately: "We had to add a 10µF cap on the 3.3V rail to fix SPI glitches." Six months later, you won't remember. Write it down now, in the ICD, with date and initials.