Validation Requirements across Domains

[rahulrazdan]

In terms of domains, the Operational Design Domain (ODD) is the driving factor, and typically have two dimensions. The first is the operational model and the second is the physical domain (ground, airborne, marine, space). In terms of ground, Passenger AVs are perhaps the most well-known face of autonomy, with robo-taxi services and self-driving consumer vehicles gradually entering urban environments. Companies like Waymo, Cruise, and Tesla have taken different approaches to ODDs. Waymo’s fully driverless cars operate in sunny, geo-fenced suburbs of Phoenix with detailed mapping and remote supervision. Cruise began service in San Francisco, originally operating only at night to reduce complexity. Tesla’s Full Self Driving (FSD) Beta aims for broader generalization, but it still relies heavily on driver supervision and is limited by weather and visibility challenges.

Transit shuttles, though less publicized, have quietly become a practical application of AVs in controlled environments. These low-speed vehicles typically operate in geo-fenced areas such as university campuses, airports, or business parks. Companies like Navya, Beep, and EasyMile deploy shuttles that follow fixed routes and schedules, interacting minimally with complex traffic scenarios. Their ODDs are tightly defined: they may not operate in rain or snow, often run only during daylight, and avoid high-speed or mixed-traffic conditions. In many cases, a remote operator monitors operations or is available to intervene if needed. Delivery robots represent a third class of autonomous mobility—compact, lightweight vehicles designed for last-mile delivery. Their ODDs are perhaps the narrowest, but that’s by design. These robots, from companies like Starship, Kiwibot, and Nuro, navigate sidewalks, crosswalks, and short street segments in suburban or campus environments. They operate at pedestrian speeds (typically under 10 mph), carry small payloads, and avoid extreme weather, high traffic, or unstructured terrain. Because they don’t carry passengers, safety thresholds and regulatory oversight can differ significantly.

Weather is a particularly limiting factor across all autonomous systems. Rain, snow, fog, and glare interfere with LIDAR, radar, and camera performance—especially for smaller robots that operate close to the ground. Most AV deployments today restrict operations to fair-weather conditions. This is especially true for delivery robots and transit shuttles, which often halt operations during storms. While advanced sensor fusion and predictive modeling promise improvements, true all-weather autonomy remains a significant technical challenge. The intersection of weather and autonomy is an active research area [1]

Another ODD dimension is time of day. Nighttime operation brings unique difficulties for AVs: reduced visibility, increased pedestrian unpredictability, and in urban areas, more erratic driver behavior. Some systems (like Waymo in Chandler, AZ) now operate 24/7, but most deployments—particularly delivery robots and shuttles—remain restricted to daylight hours. Tesla's FSD does operate at night, but it still requires human oversight. Infrastructure also shapes ODDs in crucial ways. Many AV systems depend on high-definition maps, lane-level GPS, and even smart traffic signals to guide their decisions. In geo-fenced environments—where the route and surroundings are highly predictable—this infrastructure dependency is manageable. But for broader ODDs, where environments may change frequently or lack digital maps, achieving safe autonomy becomes much harder. That’s why passenger AVs today generally avoid rural areas, unpaved roads, or newly constructed zones.

Regulatory environments further shape ODDs. In the U.S., states like California, Arizona, and Florida have developed AV testing frameworks, but each differs in what it permits. For instance, California limits fully driverless vehicles to certain urban zones with strict reporting requirements. Delivery robots are often regulated at the city level—some cities allow sidewalk bots, others ban them outright. Transit shuttles often receive special permits for low-speed operation on limited routes. These regulatory boundaries translate directly into ODD constraints.

In terms of physical domains, Ground-based autonomous systems, especially in automotive contexts, are the most commercially visible. Self-driving vehicles operate in human-dense environments, requiring perception systems to identify pedestrians, cyclists, vehicles, and traffic infrastructure. Validation here relies heavily on scenario-based testing, simulation, and controlled pilot deployments. Standards like ISO 26262 (functional safety), ISO/PAS 21448 (SOTIF), and UL 4600 (autonomy system safety) guide safety assurance. Regulatory frameworks are evolving state-by-state or country-by-country, with Operational Design Domain (ODD) restrictions acting as practical constraints on deployment.

Autonomous aircraft (e.g., drones, urban air mobility platforms, and optionally piloted systems) must operate in highly structured, safety-critical environments. Validation involves rigorous formal methods, fault tolerance analysis, and conformance with aviation safety standards such as DO-178C (software), DO-254 (hardware), and emerging guidance like ASTM F38 and EASA's SC-VTOL. Airspace governance is centralized and mature, often requiring type certification and airworthiness approvals. Unlike automotive systems, airborne autonomy must prove reliability in loss-of-link scenarios and demonstrate fail-operational capabilities across flight phases. Autonomous surface and underwater marine systems face unstructured and communication-constrained environments. They must operate reliably in GPS-denied or RF-blocked conditions while detecting obstacles like buoys, vessels, or underwater terrain. Validation is more empirical, often involving extended sea trials, redundancy in navigation systems, and adaptive mission planning. IMO (International Maritime Organization) and classification societies like DNV are working on Maritime Autonomous Surface Ship (MASS) regulatory frameworks, though global standards are still nascent. The dual-use nature of marine autonomy (civil and defense) adds governance complexity. Space-based autonomous systems (e.g., planetary rovers, autonomous docking spacecraft, and space tugs) operate under extreme constraints: communication delays, radiation exposure, and no real-time human oversight. Validation occurs through rigorous testing on Earth-based analog environments, formal verification of critical software, and fail-safe design principles. Governance falls under national space agencies (e.g., NASA, ESA) and international frameworks like the Outer Space Treaty. Assurance relies on mission-specific autonomy envelopes and pre-defined decision trees rather than reactive autonomy.

Governance also differs. Aviation and space operate within centralized, internationally coordinated regulatory systems (ICAO, FAA, EASA, NASA), while ground autonomy remains highly fragmented across jurisdictions. Maritime governance is progressing but lacks harmonization. Space governance, although anchored in treaties, increasingly contends with commercial activity and national interests, demanding updated risk management protocols.

Emerging efforts like the SAE G-34/SC-21 standard for AI in aviation, NASA's exploration of adaptive autonomy, and ISO’s work on AI functional safety indicate a trend toward domain-agnostic principles for validating intelligent behavior. There is growing recognition that autonomous systems, regardless of environment, need rigorous testing of edge cases, clarity of system intent, and real-time assurance mechanisms.

Ref:

[1] Vargas, J.; Alsweiss, S.; Toker, O.; Razdan, R.; Santos, J. An Overview of Autonomous Vehicles Sensors and Their Vulnerability to Weather Conditions. Sensors 2021, 21, 5397. https://doi.org/10.3390/s21165397