I've done this and the MySQL application does not show up for the host, and if I manually run the poller there is no <<mysql>> section in the output. I'm using the SVN release of Observium and the server I'm trying to monitor is Debian 6 with MySQL version 5.1.something. I have tested with the firewall disabled, using the user/pass set explicitly in a mysql.cnf file, and without using the one provided in the agent-local directory.
Couple of questions. Does the version of MySQL matter? Is there a good way other than the poller to check against the server to see if it's authenticating correctly? Aside from logging in using the credentials in the /etc/mysql/debian.cnf file, I've done that and it works.