It's hard to think of a browser less secure than Microsoft Edge. Other than Internet explorer of course.
In fact, from a security standpoint, it's always a good idea not to use the default browser that comes with your operating system. From the point of view of an outside attacker, it's a known target to aim for.
This isn't strictly true on Linux-based systems as the browser could be a third party browser already. Something like Firefox
or a Tor browser
Although it's not recommended to completely remove Microsoft Edge from Windows 10, you can limit what it can do on the system, and thereby limit the damage that can be done by an attacker.
Go to the Windows Start menu, scroll until you find 'Microsoft Edge'
Right-click Microsoft Edge, and in the popup menu, choose More > App Settings
In the new window that appears, turn off the following App permissions
Camera > Off
Microphone > Off
Account Info > Off
Pictures > Off
Background Apps > Off
A little further down, Click 'Terminate' to close any Microsoft Edge related programs and tasks
The 'Uninstall' option is greyed-out as it's pretty well embedded into the system (although I will write a guide on removing Edge to follow).
Use an alternative web browser - Everyone has their preference. Mine is currently 'Firefox' as it's one of the most secure and is updated far more regularly than others. Google Chrome is a close second, although there are some pretty damning exploits available for Chrome which makes it less secure (plus I'm not a fan of the extremely minimal User Interface).