Why Computer Vision Belongs in Service Dog Evaluation
Public access evaluation for service dogs has historically relied on trained human observers scoring behavior in real time. The Assistance Dogs International Public Access Test and similar rubrics ask evaluators to judge heel position, distraction response, handler focus and task execution under pressure. Human inter-rater reliability in behavioral assessment is measurably inconsistent. Scores shift with evaluator fatigue, environmental conditions and implicit expectations about breed or handler presentation.
Computer vision offers an objective, repeatable scoring layer. Pose estimation models applied to video footage can extract joint angle trajectories, proximity-to-handler distance curves, body orientation vectors and movement kinematics frame by frame. Those outputs can then be mapped against behavioral rubrics that mirror what a trained evaluator is actually looking for.
At TheraPetic® Healthcare Provider Group, our clinical and engineering teams have been investigating whether canine pose estimation pipelines can provide a structured behavioral evidence layer that supports the Licensed Clinical Doctors who review service dog documentation through platforms like verify.mypsd.org. This article presents a technical review of current model capabilities, their real-world limitations and a proposed architecture for responsible deployment.
Canine Pose Estimation Models: DeepLabCut and SLEAP Compared
Two open-source frameworks dominate non-human animal pose estimation research as of 2026: DeepLabCut and SLEAP. Both emerged from neuroscience labs studying animal movement and have since been adapted for veterinary biomechanics, ethology and increasingly for applied behavioral screening.
DeepLabCut
DeepLabCut, developed at the Mathis Lab and described in the foundational 2018 Nature Neuroscience paper by Mathis et al. (DOI: 10.1038/s41593-018-0209-y), uses a ResNet backbone fine-tuned on user-labeled keypoint annotations. For canine applications, a standard skeleton typically includes nose tip, left and right ears, neck base, spine midpoint, sacrum, tail base, and all four limb joints down to the paw. Training on as few as 200 labeled frames can achieve sub-pixel localization accuracy under controlled lighting.
DeepLabCut's transfer learning pipeline makes it relatively accessible for teams without massive labeled canine datasets. The framework exports per-frame keypoint coordinates with likelihood scores, enabling downstream filtering of low-confidence frames before behavioral metric extraction.
SLEAP
SLEAP (Social LEAP Estimates Animal Poses), developed at the Murthy and Shaevitz labs at Princeton, adds a critical capability that DeepLabCut's original architecture lacks: multi-instance tracking. When a service dog and handler are both present in frame, SLEAP can maintain separate identity-tracked skeletons across occlusion events. This matters operationally because public access test scenarios frequently place the dog in front of or beside the handler, causing partial skeleton overlap in single-camera footage.
SLEAP's bottom-up inference pipeline also performs faster than DeepLabCut's top-down approach on video batches longer than a few minutes, which is relevant when processing full public access test recordings that may run 20 to 45 minutes.
For teams choosing between them: DeepLabCut is preferable for single-animal tracking with a rich pre-trained ecosystem. SLEAP is preferable when handler-dog dyad tracking and real-time throughput are requirements. Both frameworks export to HDF5 and CSV formats compatible with standard Python behavioral analysis libraries like pandas and scipy.
Behavioral Markers Detectable in Video During Public Access Testing
Canine pose estimation is only clinically useful if the extracted keypoints map to behaviors that actually predict public access readiness. Based on the behavioral ethology literature and standard public access rubric criteria, several marker categories are extractable from video.
Handler Proximity and Heel Position
Proper heel position requires the dog's shoulder to remain at or near the handler's left hip across variable walking speeds and turns. Euclidean distance between the dog's neck-base keypoint and the handler's hip keypoint, tracked over time, yields a proximity curve. Deviation exceeding a clinically defined threshold for more than a defined percentage of the trial constitutes a scoreable failure. This calculation is straightforward once both skeletons are tracked reliably.
Gaze and Head Orientation
Head orientation vector, derived from the ear-to-nose axis, approximates the dog's attentional direction. Frequent and sustained orientation toward the handler during novel stimuli is a positive indicator. Prolonged orientation away from the handler toward a distraction source, particularly when body weight shift accompanies it, is a negative indicator associated with high arousal or task disengagement.
Arousal Posture Scoring
Body posture during arousal events can be partially scored through keypoint geometry. Tail carriage angle relative to the sacrum-spine vector, ear position relative to the skull base and overall body lowering (measured as the vertical distance between spine midpoint and floor plane) each contribute to an arousal index. High arousal is characterized by raised tail, forward ear rotation and elevated body center of mass. Low arousal or fear is characterized by tail tucking, ear flattening and crouched spine trajectory.
Arousal posture scoring from pose estimation is imprecise compared to direct behavioral coding by a certified applied animal behaviorist. Keypoint occlusion during crouching and breed-specific anatomy variation (notably between brachycephalic breeds and long-snouted working breeds) introduce error. The scoring must be treated as a supplementary signal rather than a definitive behavioral classification.
Startle Recovery Latency
Standardized distraction stimuli (dropped objects, sudden sounds, crowd movement) can be timestamped in the video. Pose estimation allows extraction of the latency between the startle event and the return of body orientation toward the handler plus normalization of tail and ear keypoints to baseline values. This recovery latency metric maps directly to public access rubric criteria around distraction response.
Task Execution Kinematics
For task-trained service dogs, the kinematic signature of specific tasks (brace position, guide halt, alert behavior initiation) may be distinguishable from pose sequence data. This is the least mature application area. Task-specific models would require annotated task execution datasets, which do not yet exist at the scale needed for generalizable classification. Research groups at Stanford HAI and affiliated veterinary behavior labs are beginning to build such corpora.
The Limits of Single-Camera Assessment
Single-camera deployment is the practical default for most public access evaluation scenarios. A handler walks through a facility, a fixed camera or a smartphone captures the interaction. The appeal is obvious: low cost, no infrastructure, immediate footage. The technical limitations are significant and underappreciated in discussions of automated scoring systems.
Occlusion and Depth Ambiguity
A single camera captures a two-dimensional projection of three-dimensional movement. When the dog moves behind the handler, sits beside them at an angle or turns laterally, critical keypoints (hip joints, tail base, contralateral limbs) fall into occlusion. DeepLabCut and SLEAP both report likelihood scores for each keypoint per frame. In single-camera footage of real-world public access scenarios, occlusion-related low-confidence frames can represent 15 to 30 percent of total frames depending on the test environment geometry and camera placement. Behavioral metrics computed across those gaps require interpolation, which introduces inference artifact.
Depth ambiguity means that a dog sitting one meter from the camera and a dog sitting three meters from the camera may produce identical 2D skeleton projections at different absolute scales. Proximity-to-handler calculations require either a known reference object in frame for scale calibration or depth sensor integration.
Breed Morphology Variation
Pre-trained canine pose models are typically trained on datasets skewed toward Labrador Retrievers and German Shepherds because those breeds dominate assistance dog training program populations. Generalization to Poodles, Golden Retrievers and especially to smaller service dog breeds shows measurably reduced keypoint localization accuracy, particularly at ear and tail landmarks where anatomy diverges most sharply from training distribution. A system that performs well on a Labrador in controlled lighting may fail on a Standard Poodle in fluorescent retail lighting.
Environmental Variation and Lighting
Public access test environments include grocery stores, airports, medical facilities and restaurants. Lighting varies from high-contrast daylight to low-lux fluorescent to mixed natural and artificial sources. Both DeepLabCut and SLEAP show performance degradation under lighting conditions not represented in training. Domain adaptation techniques, including synthetic data augmentation and test-time augmentation, can partially close this gap but add engineering overhead and are not yet standard in off-the-shelf deployment pipelines.
Multi-Camera Triangulation and Clinical Integration
The most technically rigorous approach to video-based public access assessment uses synchronized multi-camera rigs enabling 3D skeleton reconstruction through triangulation. OpenCV's camera calibration toolchain combined with DeepLabCut's 3D triangulation module (anipose, described in Karashchuk et al., eLife 2021, DOI: 10.7554/eLife.61909) allows extraction of true metric-unit body coordinates. Proximity calculations, arousal posture scoring and startle recovery latency metrics computed from 3D data are substantially more reliable than their 2D equivalents.
A practical multi-camera deployment for public access evaluation might use three to four wide-angle cameras mounted at ceiling height covering a defined evaluation corridor, synchronized via NTP or hardware trigger. The infrastructure cost is non-trivial: camera hardware, calibration labor, compute for triangulation and storage for synchronized 4K streams across a 30-minute evaluation. For high-volume service dog training programs, that cost amortizes over hundreds of evaluations per year. For individual handler verification, it does not.
The clinical integration question is as important as the technical architecture. Pose estimation output should feed into a structured behavioral scoring dashboard reviewed by a Licensed Clinical Doctor or a certified applied animal behaviorist, not interpreted by the model alone. The model surfaces quantitative behavioral signals. The clinician applies diagnostic judgment, contextualizes breed-specific norms and makes the final determination. This is the hybrid model TheraPetic®'s engineering and clinical teams consider the only defensible deployment pattern for documentation that carries legal weight under current federal access law.
Applying Pose Estimation to Verified Service Dog Documentation
TheraPetic®'s HANK AI infrastructure, accessible through servicedog.ai and integrated with the verification layer at verify.mypsd.org, currently uses clinical intake screening, handler self-report and Licensed Clinical Doctor review as the primary documentation pathway. Video behavioral assessment is not yet part of the standard intake flow.
The engineering roadmap under active internal evaluation would add an optional video submission module where handlers upload a standardized 10-minute public access simulation recorded from two camera positions following a structured protocol. HANK AI would run a DeepLabCut or SLEAP pipeline on the uploaded footage, generate a behavioral marker report covering the metrics described above and present that report to the reviewing Licensed Clinical Doctor alongside the standard intake data.
The video module would not replace clinical review. It would give the reviewing clinician a structured, objective behavioral dataset that supplements handler self-report. Handlers who self-report excellent public access behavior that the video analysis contradicts would trigger a follow-up clinical interview rather than automatic denial. The data informs the clinician. It does not supplant the clinician.
Data governance for video submissions would follow the frameworks outlined through mydatakey.org. Video footage containing both handler and dog presents privacy considerations beyond standard clinical text data. HIPAA-aligned deidentification of handler biometric information would be required prior to any footage retention or model retraining use.
Algorithmic Fairness and Clinical Oversight Requirements
Automated behavioral scoring systems applied to access verification carry real stakes. An inaccurate or biased model could contribute to denial of documentation for a legitimately trained service dog. That outcome has direct legal implications under the Americans with Disabilities Act and the Fair Housing Act, both of which protect the rights of individuals with disabilities to be accompanied by trained service dogs and support animals in applicable settings.
Algorithmic fairness analysis for canine pose estimation systems should evaluate whether scoring accuracy is consistent across dog breeds, coat colors, handler mobility patterns and filming environments. Demographic parity and equalized odds metrics, commonly applied to human-facing classification systems, have direct analogs in this context. A model that achieves 90 percent behavioral scoring accuracy on Labradors and 65 percent accuracy on Poodles is not clinically deployable without breed-stratified correction.
The Partnership on AI's guidelines on responsible AI deployment in high-stakes decision contexts apply to this use case. Stanford HAI's fairness in clinical AI frameworks, while focused on human health applications, offer methodological templates for bias auditing that can be adapted for veterinary behavioral AI. Any deployment of canine pose estimation in documentation workflows should include pre-deployment bias auditing, ongoing performance monitoring stratified by breed and environment category and a clear human override pathway at every decision point.
TheraPetic®'s position, consistent with the clinical ethics framework articulated by Dr. Patrick Fisher's team, is that no automated behavioral scoring system should carry determinative weight in service animal documentation decisions as long as breed generalization error and single-camera occlusion rates remain at current levels. The technology is a powerful clinical support tool. It is not yet a clinical decision maker.