I have a non-linear output equation of an amplifier vo = a1*vi + a2*vi^2+a3*vi^+H.O.T... and I want to implement the equation in ADS. How would I do that?

There are many possible approaches, but I would start by using a SDD (symbolically defined device.) Please see the SDD_Examples workspace in the Tutorial folder.

Could you please me more details about how to input the vo equation I have? The examples do not seem clear to me. I guess I am not sure about the Vi in the Vo equation.

For your specific problem, where an output voltage should be set as a polynomial function of an input voltage, I would use the 'NonlinVCCS' component, which can be found under 'Eqn Based-Nonlinear' in the component palette

There are many possible approaches, but I would start by using a SDD (symbolically defined device.) Please see the SDD_Examples workspace in the Tutorial folder.