An APV layout was one of my dreams too, but no it cannot be done
However, you do suggest the way forward…
You need to make 'Sprung' points
In theory … Duplo points (which are correctly called Lazy points) can be modified. Some kind of spring or elastic band needs to be added to keep the points in one position while allowing the train to pass over undisturbed. The points move just enough to let the train pass before springing back.
In practice it’s difficult to get right. But I feel sure it could be done.
Here is a circuit using only Sprung points. The spring ensures that the train only exits on the straight line, never the curved ‘branch’ line
The train joins the main loop and then visits each ’sub layout’ 1, 2, 3 in turn. The train always returns from each sub layout back along the same track. There is even a bell to alert the operator to the start of the next loop :-)
I think you can see the similarities with a computer program, where each sub layout can then ‘call’ other sub layouts etc etc. So you end up with a multiple branching tree-like structure
So this technique will allow you to build large APV track layouts...