The primary disadvantage of your loan not being a QM is that you do not receive the extra level of protection afforded to QMs. If your HPML is a higher priced covered transaction (HPCT) it is not eligible for the safe harbor protection, but it would be eligible for the presumption of compliance protection. Also a QM is exempt from the appraisal requirements that apply to first lien HPMLs.
If the loan does not achieve QM status, then in order to make the loan it must meet the eight factors.