I don’t think the duplex would preclude you from using the detached structure exemption since it is a residential structure used for personal, family, household purposes. Is the barn used for an agricultural purpose? If so, that would disqualify it from the exemption. Also consider the bank may want/need to insure the barn to protect the collateral or the customer may choose to insure it to protect their property.
The notice would be required.
“Notice requirement. When an FDIC-supervised institution makes, increases, extends, or renews a loan secured by a building or a mobile home located or to be located in a special flood hazard area, the FDIC-supervised institution shall mail or deliver a written notice to the borrower and to the servicer in all cases whether or not flood insurance is available under the Act for the collateral securing the loan.”