Octopus Overlords

Here are my specs as of right now:

Core i5 4590 (1160 socket)
GTX 1060 3GB
16GB RAM

I don't run into problems with any games UNLESS I'm playing Retroarch, and then my system starts chugging when I'm running a PSX or DS emulator. What do y'all think? Am I way behind the times?

Assuming you are running 1080p I would guess you are mostly fine.

I am running an i5-4460 and a Geforce 970 and 16G. I think things by and large run fine for me. Your CPU is a bit quicker and the 1060 is a bit faster but has slightly less VRAM than my 970 - although I remember that the 4th Gig had some limitations with the 970.

Whether or not you need to upgrade is your call. As far as emulation goes in theory CPU is more important, but video card becomes important for more recent consoles.

In general, the best upgrade you can make for gaming is your video card. My gaming rig is 4 years old, and I may think of doing a video upgrade sometime this year to eek out a few more years of gaming life out of it before retiring it to solely being a Plex server. I have no plans to venture beyond 1080p in the near future. I would probably need 2 TVs to die before I worry about 4K and then I would defintely be building something new. I am hoping the prices for good 4k video cards will have fallen down to more mainstream prices by then.

Oops my gaming rig is 5 years old! (Still in this awkward new year mode where I can't figure out what year it is, LOL.) I just read an article from last year about how the 970 is still pretty effective at 1080p gaming. Looks like most games from a half year ago or so were still pretty playable at 1080p with dips in the 40s/50s FPS. Definitely more than good enough for me for now.

If it's emulation bringing things down that implies a CPU limitation. 4 cores/4 threads is not enough these days for CPU heavy tasks. You could move up to an i7 4770 assuming you could find one at a decent price. If it were me I'd use a tool like MSI Afterburner to monitor CPU and GPU usage while running the emulator, just to verify how much of each is being used. The other thing I'd do is check out suggested system requirements for the games you are trying to emulate. Demands may vary, so one game might require a lot more CPU or GPU than another. You might be better off with a new 4 core/8 thread CPU than an older i7, for example. Or you might discover that an older i7 is enough.

tbh that setup should NOT be chugging when emulating PS1/DS games...

also, are did you install Linux (RetroArch) in its own partition on your SDD? you definitely shouldn't be having problems then.

Should be fine for 1080p and below. Wouldn't do super refresh, but should not chug either.