Ok, just to circle back, I've made some changes:
1: Set the number of poller-wrapper.py processes to 25, this greatly speeded up things. Thanks!
2: I still noticed I had a few devices that would take 8-12 minutes to finish a poll. "ps kstart_time -ef | grep "wrap\|poller.php" will help identify when processes started and what devices were taking so long. Incidentally this is found in the "observium.log" file as well, but this is a bit more real time. The devices that were taking a really long time have only a handful of interfaces but around 2500 vlans and associated SVI/VE interfaces. The hardware doesn't supply traffic stats on these interfaces so I disabled lots of polling and discovery modules. There are lots of ARP entries on these devices as well, I don't really require those so I disabled those modules as well. Looked through the "includes/defaults.inc.php" and put a lot of the stuff I didn't need or want into "config.php" and set to disable. The devices still take around 3 minutes to finish, but at least its not 5!
I think things are much better now, thanks for the help!
--chip