What if farms were 1x1 and box-draggable?

I've always wondered if farms can be redone to be 1x1 tiles, and draggable like walls, but in a square field.

Each tile would ripen up to 3 food, and refill at a rate that's 1/10th of the villager farm gather rate, so each villager can attend to roughly 9 tiles of farmland, but it can be in any shape that's convenient. The farms plants would get visually bigger and more numerous as they ripen from 0 to 3 food, and this amount would be drawn from a total before reseeding is needed. Villagers would then move to a nearby tile with a preference for the ripest one, and work their way around the farm tiles as they slowly grow back.

Farms have always been kind of odd, such that villagers favor the left side of them, and they don't fit well with either mills or town centers.

Is there any way something like this could work without causing problems?