Adding routerProvider to Refine v4 errors with useLocation being used outside of a router

I am trying to upgrade my project to v4 and I've been having lots of issues. Right now trying to use the new router immediately fails if I try to add the routerProvider to the Refine component

I have imported the routerProvider from @refinedev/react-router-v6 and BrowserRouter from 'react-router-dom' I don't understand why its not working