Yes, a registered legal entity such as an LLC is the foundation. The government contracts with your business, not with you personally. Your entity is first registered with your state, and that entity is then registered with SAM and the other federal, state, and local vendor systems that give your company its identifiers.
You do not need the LLC before joining the program, though. Members regularly join without any formal business setup and form their entity as part of their roadmap. The program can coach you to file it yourself, which is fairly simple, or handle the filing for you as a done-for-you service.
It is worth talking with your mentor before filing so the business is structured the right way from the beginning, especially if ownership structure matters for certifications you may want later. The main outside cost is your state filing fee, which varies by state.