There was a paper from Neel Nanda, where they were able to show that the neurons actually learnt the algorithm. Which is not the case for most cases.

Heuristics is only efficient in search, because it reduces number of pathways, it is not more efficient than if you know the exact algorithm to reach a certain step.

In order to do basic algebra, doing direct arithmetic computations are far more effective than using any heuristics.