Training a lying-on-the-ground detector on synthetic data only: D-FINE vs RF-DETR vs YOLO on real footage
We trained small detectors only on synthetic CCTV images and scored them on two sets of real, staged footage they had never seen. This is the method, the full round-1 results, the training recipe that made the difference, and what we do not claim yet.
In short: trained only on 900 synthetic images, D-FINE Small found 97.4% of real lying frames at a 5% false-alarm rate in a staged home it had never seen, and 89.9% of fall clips in a second home. D-FINE Nano trained on real footage from one room was perfect in that room but caught 24.1% of fall clips next door. YOLO26 with its default recipe did not transfer at all.
The task: is someone lying on the ground?
The model answers one question per person in the frame: is this person lying on the ground or floor, or are they sitting, kneeling, standing or walking? Lying on the ground covers the outcomes a CCTV or security operator cares about: a collapse, a fall, an assault. The positives include end-on lying, where the person lies along the camera's line of sight and the box is short and foreshortened rather than flat and wide.
Two things are deliberately out of the test. The falling motion is not scored. An alarm a fraction of a second earlier changes nothing in practice; what matters is reliably seeing that someone is down, and counting the blurred mid-fall frames would make every model look worse without telling you anything useful. Lying in bed or on a sofa is out of scope. Lying on furniture is normal behaviour at home, so a product that watches beds needs its own negatives and its own test. Mixing the two would blur what this model is for.
So this guide talks about a detector of a person lying on the ground or floor. Some of the test data comes from fall datasets, and we use their fall clips as positives, but the model is not a general fall detector.
The training data
All synthetic-only models were trained on Simuletic CCTV Fall Detection 1K: a fixed 900 / 100 train-val split, with 178 confuser images of people sitting, kneeling, crouching, squatting, standing and walking. The confusers are the point: the hard part of this task is not finding people lying flat in the middle of a floor, it is not alarming on everyone who sits down.
For comparison we also trained on real footage. Real only uses CAUCAFall subjects 1 to 4 for training and subject 5 for validation: 1,382 frames, every fifth frame. Mixed is Fall 1K plus the same real frames.
Two real tests
Test 1: CAUCAFall subjects 6 to 10. No model was trained on these people. 1,923 frames: 615 lying and 1,308 not lying. We leave out the second either side of each fall onset, and frames the dataset relabels “no fall” while the person is still on the floor. It is the same room and camera as the real training subjects, so for the real-only and mixed models this is an in-domain test.
Test 2: GMDCSA-24. 160 clips from a different home with four people, sampled at 2 frames per second. Positives are the fall clips, where the person ends on the floor. The 41 clips of lying in bed or on a sofa are left out of scoring. A clip's score is its second-highest frame score, so one stray frame cannot trigger it on its own. This is the out-of-domain test for every model, and the fair one for comparing synthetic against real training data.
Metrics
- Frame AP on Test 1 for lying versus not lying.
- Lying frames caught at a fixed false-alarm budget: the share of lying frames found when 5% or 1% of not-lying frames may alarm. These are operating points, closer to how a deployed alarm is tuned than AP is.
- End-on lying caught at 5%: the same, for the foreshortened cases only.
- On Test 2: clip AP, and the share of fall clips caught at a 5% and a 10% clip false-alarm rate.
Training ran on a MacBook M4 Pro. We do not report speed: latency was measured while other training jobs were running, so the numbers would not mean much.
Results, round 1
Every row is a single training run except D-FINE Nano, which we trained three times to see the run-to-run spread. T1 is Test 1, T2 is Test 2.
| Model | Train data | Params | T1 AP | T1 @5% | T1 @1% | T1 end-on @5% | T2 clip AP | T2 falls @5% | T2 falls @10% |
|---|---|---|---|---|---|---|---|---|---|
| D-FINE Small | synthetic | 10.2 M | 0.985 | 97.4% | 83.1% | 90.1% | 0.984 | 82.3% | 89.9% |
| D-FINE Nano (seed 0) | synthetic | 3.7 M | 0.972 | 92.7% | 74.8% | 64.0% | 0.953 | 59.5% | 87.3% |
| D-FINE Nano (seed 1) | synthetic | 3.7 M | 0.937 | 90.2% | 51.7% | 54.0% | 0.967 | 67.1% | 83.5% |
| D-FINE Nano (seed 2) | synthetic | 3.7 M | 0.971 | 91.9% | 83.6% | 61.3% | 0.961 | 65.8% | 83.5% |
| D-FINE Nano | real only | 3.7 M | 1.000 | 100% | 100% | 100% | 0.850 | 22.8% | 24.1% |
| D-FINE Nano | mixed | 3.7 M | 1.000 | 100% | 99.5% | 100% | 0.958 | 57.0% | 89.9% |
| Two-stage (MobileNetV4 crop classifier) | synthetic | 6.3 M | 0.895 | 77.6% | 71.2% | 6.3% | 0.978 | 84.8% | 88.6% |
| Two-stage | real only | 6.3 M | 0.881 | 77.1% | 39.2% | 73.9% | 0.914 | 49.4% | 54.4% |
| Two-stage | mixed | 6.3 M | 0.948 | 91.2% | 80.6% | 73.9% | 0.973 | 86.1% | 92.4% |
| RF-DETR Nano | synthetic | 30.2 M | 0.898 | 88.3% | 16.1% | 61.3% | 0.955 | 46.8% | 93.7% |
| YOLO26n (reference, AGPL) † | synthetic | 2.5 M | 0.449 | 19.4% | 19.4% | 1.8% | 0.755 | 7.6% | 13.9% |
| YOLO26n, feature layers frozen † | synthetic | 2.5 M | 0.813 | 73.0% | 41.8% | 40.5% | 0.884 | 26.6% | 26.6% |
| YOLO26n † | real only | 2.5 M | 0.866 | 70.9% | 13.7% | 82.9% | 0.746 | 10.1% | 15.2% |
| YOLO26n † | mixed | 2.5 M | 0.940 | 87.8% | 39.4% | 82.9% | 0.801 | 20.2% | 31.6% |
| YOLO26s (reference, AGPL) † | synthetic | 10.0 M | 0.380 | 12.5% | 11.1% | 0.0% | 0.668 | 2.5% | 11.4% |
† YOLO26 is AGPL-3.0. These rows are a reference point only; we do not ship YOLO models (see YOLO licensing vs dataset licensing). Real-only and mixed models are in bold in the train-data column; Test 1 is in-domain for them.
What the results show
- Synthetic data generalised; real data from one place did not. D-FINE Nano trained on real CAUCAFall footage is perfect on more CAUCAFall footage, same room and camera. In the other home it catches 24.1% of fall clips at a 10% clip false-alarm rate (clip AP 0.850). The same model trained only on synthetic images catches 83.5% to 87.3% across three runs (mean 84.8%, clip AP 0.953 to 0.967). This is the main result.
- Synthetic plus real gets both. Mixed D-FINE Nano is perfect in-domain and catches 89.9% of fall clips in the new home. The two-stage design shows the same pattern: 54.4% of fall clips real-only, 92.4% mixed.
- D-FINE Small is the strongest synthetic-only model. 97.4% of lying frames at a 5% false-alarm rate on Test 1, 90.1% of end-on lying, and 89.9% of fall clips on Test 2 at the 10% setting.
- Run-to-run spread is real. Across three D-FINE Nano runs, Test 1 AP ranges from 0.937 to 0.972 (mean 0.960), and the strict 1% setting varies most, from 51.7% to 83.6% (mean 70.0%). A single run, or the best of several, would mislead you; we report the mean and the range.
- The fine-tuning recipe matters as much as the architecture. Freezing YOLO26n's pretrained feature layers lifts its Test 1 AP from 0.449 to 0.813. D-FINE's lower learning rate on the same layers is part of why it transfers.
- Bed lying is a separate problem. It is out of scope and left out of Test 2. For information: at the 10% setting the synthetic-only models alarm on 24% to 32% of the bed clips, so a product for beds would need its own negatives.
Two smaller patterns are worth knowing. End-on lying is the weak spot for the small models: D-FINE Nano catches 54.0% to 64.0% of it (mean 59.8%), and the synthetic-only two-stage crop classifier only 6.3%. And RF-DETR Nano has the best Test 2 score at the 10% setting (93.7%) but the weakest strict 1% score on Test 1 (16.1%) of the non-YOLO synthetic models, so its ranking depends on where you set the alarm threshold.
The fine-tuning recipe
D-FINE: 60 epochs, with the pretrained image-feature layers training at a learning rate 10x lower than the detection head. YOLO26: 100 epochs with the Ultralytics defaults. Crop classifiers for the two-stage design: 40 epochs.
The YOLO26 result is the useful warning. With the default recipe, every layer trains at full speed on the new data. On 900 synthetic images, that overwrites the real-world features the model learned in pretraining, and what is left is tuned to the look of the training set. On real footage it collapses: 0.449 AP on Test 1 for YOLO26n, 0.380 for YOLO26s. Freezing the pretrained feature layers keeps those real-world features and lets only the detection layers learn the new task, and Test 1 AP goes to 0.813.
The general rule for synthetic-only training: protect what pretraining learned from real images. Freeze the early layers or slow them down, and teach the new class in the layers that decide. Our D-FINE recipe does it with the split learning rate; with YOLO's default recipe you have to change it yourself.
Caveats
- Both real tests are staged, small and indoor, with few actors. Nothing here measures night, distance or crowds.
- Test 1 is in-domain for the real-only and mixed models, so their perfect Test 1 scores overstate them. Test 2 is the fair cross-domain test.
- Every row is one training run except D-FINE Nano. Latency was measured while training ran and is not reported.
- RF-DETR was trained on synthetic data only in round 1; there is no RF-DETR real-only or mixed result here.
What comes next: round 2
Round 2 trains D-FINE only, Nano and Small. The training data is Fall 1K plus 400 targeted synthetic images aimed at the gaps round 1 found, such as end-on lying, people partly hidden by furniture, and sitting-down look-alikes. Control runs on Fall 1K alone show what those 400 images add, and each configuration runs with several seeds.
The tests widen too: a public-space slice from CHAD (fixed, elevated cameras over a real car park) and false alarms per camera-hour on MEVA (elevated campus CCTV, CC BY 4.0, nobody lying), both subject to licence checks, plus speed measured on an idle machine.
We release the models when round 2 meets this bar:
- D-FINE Nano catches at least 80% of end-on lying at 5% false alarms (three-run mean). Today it is 54% to 64%.
- On Test 2, at least 85% of fall clips at the 10% setting for the Nano mean, and 90% for Small.
- The strict 1% setting varies by no more than 15 points across runs. Today it is 32 points.
- Nothing drops more than 2 points compared with round 1.
- The public-space test is scored and reported, whatever it says, together with false alarms per camera-hour.
Then the D-FINE Small and Nano weights go on Hugging Face under Apache-2.0 with the round-2 scorecard. Until then the weights are not released; the model page for D-FINE Small, person lying on the ground and the models overview track where it stands. For the short version of this study, read the case study or the blog post The real-footage model was perfect at home and lost in the neighbour's house.
Test data and licences
Both real test sets are used under their published licences, and our thanks go to their authors:
- CAUCAFall by Eraso, Muñoz, Muñoz and Pinto (Universidad del Cauca), Mendeley Data, CC BY 4.0.
- GMDCSA-24 by Alam, Sufian, Dutta, Leo and Hameed (2024), MIT licence.
- Models: D-FINE and RF-DETR (Apache-2.0); YOLO26 (Ultralytics, AGPL-3.0) as a reference only.
Want to run the same test on your own cameras? The synthetic training set is for sale under a research or commercial licence (see pricing), and we build custom sets for specific camera views.
Common questions
Can a model trained only on synthetic images find a person lying on the floor in real footage?
In our round-1 tests, yes, in staged indoor homes. D-FINE Small trained only on synthetic images caught 97.4% of lying frames at a 5% false-alarm rate on CAUCAFall subjects 6 to 10 and 89.9% of fall clips at a 10% clip false-alarm rate on GMDCSA-24. Public spaces, night, distance and crowds have not been measured yet.
Why did the model trained on real footage do worse in a new home?
Its real training footage came from one room and one camera. It was perfect on new people in that room, but on GMDCSA-24, a different home, it caught 24.1% of fall clips at a 10% clip false-alarm rate. The same D-FINE Nano trained only on synthetic images caught 83.5% to 87.3% across three runs.
Why does YOLO26 score so low when trained on synthetic data?
With the default Ultralytics recipe, training on synthetic images overwrites the real-world features YOLO26n learned in pretraining: it reached 0.449 AP on the in-domain real test. Freezing its pretrained feature layers lifted that to 0.813. YOLO26 is AGPL-3.0, so we report it as a reference only and do not ship it.
Why is the falling motion itself not scored?
The question the model answers is whether someone is down, not whether they are falling. An alarm a fraction of a second earlier changes nothing in practice, so we leave out the second either side of each fall onset and score the lying state against sitting, kneeling and standing.
When will the model weights be released?
After round 2 passes our release bar, which includes D-FINE Nano catching at least 80% of end-on lying at 5% false alarms (three-run mean). D-FINE Small and Nano weights will then go on Hugging Face under Apache-2.0 with the round-2 scorecard.
Train on the same data
Fall 1K is the synthetic set every synthetic-only model above was trained on: YOLO and COCO labels, a fixed 900 / 100 split and look-alike negatives. Start with the free sample, or tell us about your cameras.