KEXTs are Kernel Extensions, and I realised that a program that I installed in the past (SteerMouse) had left its KEXT behind even though I uninstalled the program. You can see the list of KEXTs in your system at /System/Library/Extensions/, but do note that Mac OSX only loads whichever KEXT is needed.
As you can’t just delete the KEXT from within Finder, I had to find out how to remove it. Turns out it’s just one command in Terminal:
rm -rf /System/Library/Extensions/SteerMouse.kext
My KEXT is named SteerMouse.kext, but you can replace that with the KEXT name of your choice. Once you’ve got it deleted, reboot your system and the KEXT will be gone.