Will they really grow taller when they did not produce much fruit?
Just realized I didn't answer the question.
As survival of trees depends heavily on height and not having sunlight blocked by taller plants, growth is a top priority.
Most wild trees have proportionately small root bases with most energy and development trending towards trunk and branch development for height and survival.
The goal of farmers is to produce a tree with opposite proportions, a huge root base with minimal branches.
The standard to achieve this is to selectively prune branches from the tree, reducing the amount of upkeep needed to maintain the trees existence.
Pruning branches, frees up energy, allowing trees to devote greater energy towards root growth and development.
The more roots a tree has, the more nutrients, minerals and vitamins it can pull from the ground.
This greater energy can be directed towards fruit production or tree growth depending on what type of fertilizer, mulch, compost, soil amendments a tree is fed.
Whether trees grow taller can be dependent on available carbon. Carbon comprises the hard element in wood and is a necessity for growth. But TBH trees can be weird and have their own strange quirks depending on weather conditions, etc. I have a mango tree that produced zero flowers this year that seemed to put its energy into root development rather than height. Stuff can be strange.