GitLab and the Agentability Blind Spot: Why Most SaaS Products Fail Transparency and Shadow-UI Tests

As AI agents move from experimental demos to production workflows, a critical question emerges: can these systems actually operate the software we've spent decades building for humans? A new audit of the top 100 SaaS products suggests the answer is largely no—and the reasons why reveal fundamental assumptions about interface design that the industry now needs to reconsider.

The Agentability Project recently completed an assessment of 100 leading SaaS products, measuring their compatibility with AI agent operation across eight Agent Factor Engineering (AFE) principles. The results paint a picture of an ecosystem built almost exclusively for human interaction, with little consideration for programmatic access beyond traditional APIs.

The Agentability Landscape

Agentability measures how well AI agents can operate software interfaces on a scale of 0-100, evaluating products across eight core principles that enable autonomous interaction. The audit revealed an average score of just 38.3 out of 100 across the sample.

These numbers reflect a transitional moment. While nearly a quarter of products demonstrate baseline agent compatibility, the majority occupy a middle ground—not entirely hostile to agent interaction, but far from optimized for it. The eight AFE principles measured include machine readability, transparency, shadow-UI avoidance, defaults, control, chunking, status reporting, and clean handoffs. Each principle addresses a specific friction point that agents encounter when attempting to operate software designed for human users.

Two Systemic Failures

While scores varied across the eight principles, two stand out for their near-universal failure rates. These aren't edge cases or advanced features—they represent foundational design patterns that the vast majority of products have simply not implemented.

Transparency: The 83% Zero-Score Problem

Transparency in the AFE framework refers to a product's ability to expose its capabilities, requirements, and constraints in machine-readable formats. This includes publishing structured documentation about available actions, parameter requirements, rate limits, permission models, and state dependencies. When transparency is absent, agents must guess at capabilities, leading to brittle integrations and unpredictable failures.

The near-universal failure on this principle suggests that most product teams haven't considered making their interface contracts programmatically discoverable. Human users navigate software through exploration and visual cues; agents require explicit declarations of what's possible and what's required.

Shadow-UI Avoidance: The Hidden Interaction Tax

Shadow-UI refers to interface elements that appear, disappear, or change based on hover states, click sequences, timing conditions, or other ephemeral triggers. Context menus, tooltip-based controls, hover-to-reveal actions, and dynamically loaded content all fall into this category. While these patterns create clean interfaces for human users, they present fundamental challenges for agents that lack mouse hover capabilities or must operate within strict action-sequence constraints.

The prevalence of shadow-UI patterns reflects an interface design philosophy optimized for spatial economy and progressive disclosure—both valuable for human users working within fixed screen dimensions, but obstacles for agents that benefit from persistent, fully-exposed control surfaces.

GitLab in Context

GitLab's agentability profile provides a concrete example of these broader patterns. As a DevOps platform with substantial API coverage, GitLab demonstrates strengths in certain areas while reflecting the industry-wide weaknesses in transparency and shadow-UI avoidance. The platform's detailed agentability assessment shows how even technically sophisticated products with robust API layers can struggle with agent-facing interface design.

GitLab's position within the Agentability Index illustrates that traditional API coverage doesn't automatically translate to agent operability. Agents increasingly need to operate software through the same interfaces humans use—clicking, filling forms, navigating workflows—either because APIs don't cover all functionality or because the agent's task requires understanding and manipulating the user-facing interface itself.

What These Numbers Mean for Product Teams

The audit results don't suggest that current products are poorly designed. Rather, they reveal that a new set of design requirements has emerged—one that most teams haven't yet incorporated into their development processes. The average score of 38.3 indicates that products possess partial agent compatibility, often unintentionally, through design choices that happen to benefit both human and agent users.

The wide variation in principle scores offers a roadmap. Machine readability scored highest at 71 on average, suggesting that semantic HTML and proper labeling practices are relatively widespread. Control (48), chunking (45), status (47), and clean handoffs (40) occupy a middle range, indicating mixed implementation.

The concentration of failures in transparency and shadow-UI avoidance points to specific, addressable design gaps rather than requiring complete interface redesigns.

Concrete Next Steps

Product teams looking to improve agentability should consider a phased approach:

• Audit shadow-UI patterns: Identify interactions that require hover states, hidden menus, or timing-dependent sequences. Consider providing alternative access paths that expose these controls persistently or through explicit navigation.
• Publish capability contracts: Create machine-readable documentation that declares available actions, required parameters, and permission requirements. This doesn't replace human documentation—it supplements it with structured data agents can parse.
• Instrument state exposure: Ensure that application state, loading conditions, and error states are exposed in ways agents can programmatically detect, not just visual indicators.
• Test with agent scenarios: Include agent-based interaction patterns in testing protocols. Can a process be completed without hover interactions? Are loading states detectable programmatically?

Teams can run a free audit to establish a baseline and identify the highest-impact areas for improvement within their specific product context.

The transition to agent-compatible interfaces doesn't require abandoning human-centered design. Rather, it demands an expansion of that framework to accommodate a new category of user—one that interprets interfaces through structured data rather than visual composition, and that benefits from explicitness over inference. The 100-product audit reveals an industry at the beginning of this transition, with clear patterns of where the work needs to focus.