Tech News
← Home  ·  All topics

Robot Planning

1 GoKawiil brief on this topic · RSS for robot planning

Researcher runs 600 tests exposing flaws in JEPA world-model benchmarks

An independent researcher built a memory-graph planner (LeWM) and tested it against the standard Push-T benchmark, finding that 84% of the benchmark's test goals already closely matched states recorded elsewhere in the training demonstrations, making the task easier than it appears. The researcher also found results were highly sensitive to sample size and random seed, with success rates on the official 50 episodes dropping from 68% to 44% on a larger 250-episode set, and 38 episodes flipping outcome purely from a changed seed.