No. You will never be asked to justify your use of the debit card associated with your HSA. This is a personal bank account, and you are the only one responsible for its use. Be aware, however, in case of an IRS audit you may be asked by the IRS to provide proof that any funds spent out of your HSA were for a qualified medical expense. For this reason, you should keep all receipts and EOBs with your tax records.
Find more HSA-related FAQs here: https://www.optumbank.com/tennessee/customer-support/faqs/hsa-faqs.html.