[Observium] APCu Caching not working?

I have noticed that when I enable webUI caching, that the page loads don’t speed up. It regularly takes 3 seconds to render a page, although the tool tip at the bottom shows APCu caching is enabled. However, if I force the cache driver to “files”, then I see a huge speed up (page loads drop to about 100-200ms).

Is there some extra config to get APCu working correctly? I didn’t see anything in the docs, but I could have missed it.