# Solving equations with composed constrained functions

I was lately curious about an iterative approach that would solve maths equations containing composed functions with contraints.

For example, if I have the following equation:

$$f(g(h(w))) = 0 text{, with } w = begin{pmatrix} a_{11} & 0 & ldots & a_{1n}\ 0 & a_{22} & ldots & a_{2n}\ vdots & vdots & ddots & vdots\ 0 & 0 &ldots & a_{nn} end{pmatrix}$$

Along additional constraints on the $$3$$ functions like
$$f < g$$;
$$h > 2 * g$$; and
$$f, g,h$$ not constant

The goal is to find the $$3$$ functions expressions given a specific matrix $$w$$ and the constraints.

What approach would be the most convenient to find a solution or solutions to this problem ? I was personally thinking about using reinforcement learning (machine learning) where each time a solutions is chosen, a positive or negative reward will be attributed to the solution generator.

Thank you