By default, virt-manager connects to the system session. You can connect to the user session by choosing File → Add connection → Hypervisor: QEMU/KVM User session → Connect.