How can I close the "Navigation to the webpage was canceled" window?

Not Sure this will Fix the issue but worth giving it a go, if it is a pop-up window over the top of your main window or a modal/iframe try right clicking on the aspect which you wish to close, click on inspect element, this will bring up a console window at the bottom of the screen with the element highlighted in the Node Tree pane, hover over the element that is already highted and then right click on it and select "Delete Node".. However if this is a full size window or the entire Internet explorer window this will not work, but I use this often to get rid of annoying Adverts, Models, or cookie prefferance windows.. Try it and comment if it works. Also can you add which website your trying to navigate to so we could replicate what your recieving. It may just be that the website is either Not valid, the server is down or There page is not set-up properly...

Example image:
Your browser may display differently,
but its the same principle.

Alternatively, Rather than restarting your entire Pc you can right click on the settings icon at the bottom left of your screen, open up task manager & end the Internet Explorer procces, by right clicking and selected end task, end proccess or terminate task. Saves the hassle of rebooting.

If it is freezing the computer completely and you cannot move the cursor to the bottom left settings menu, Pressing Ctrl+Alt+Del Usually works to bring up options of viewing task manager. Let me know if any help..