I've discovered how to change the taskbar icon name.
Right click on the icon, then, from the menu that pops up, right click on the icon name, then, from that menu that pops up, click on properties. In the shortcut properties dialogue, select the General tab and next to the icon is the name box, edit that to what you want the new name to be, then click OK.
Hovering over the taskbar icon might not show the new name at first, it does not for me but that may be a Windows 11 bug. If I hide then show the taskbar, or re-launch windows explorer or log out then back in, the new name shows up.