A fresh install of WordPress is a bit like an old diesel truck, it smokes a lot and gets pretty crappy mileage, and the locks are safe but from those with wire clothes hangers. It is a great script nonetheless and with a few teaks, can perform like a custom hybrid hotrod which gets great mileage and has a great security system.
This site runs on WordPress but I’ve done a few things I strongly recommend no matter how you host your installation be it a shared host, VPS or even dedicated server.
Recommended Plugin #1: W3 Total Cache
This is one plugin I’ve recommended time and time again to people both in site consultations and for end-users in technical support. This is one plugin which offers caching options on multiple levels with excellent flexibility and configurations. When setup properly this nifty plugin reduces the number of renders PHP needs to perform for your site coding and queries to the database among other things.
Recommended Plugin #2: DB Cache Reloaded Fix
WordPress stores a lot of information in the database, posts, configuration data, comments, pings and tracks, revisions, you name it. This plugin has the simple task of remembering the results of those queries for X minutes so if say 50 people hit the first page of your blog, the queries which build that page only need to be run once and the cached results are used to serve the rest of those people.
Recommended Plugin #3: Akismet
WordPress has comments and just like forums, open public areas invite spammers. With this plugin you can eliminate spam comment build up which will slow your blog down over time and make your database grow in size with links to content you otherwise wish you never knew existed.
Recommended Plugin #4: Revision Diet
WordPress automatically stores a changelog of each and every page and post you make and if you say make several edits or revisions or more before publishing, that builds up post after post. I highly recommend this plugin to maintain a smaller and more efficient database because after all, do you really need ten or more revision versions of a post on anything?
Recommended Plugin #5: Bulletproof Security
This is one plugin I recommend for security alone, this plugin has the sole purpose of securing your installation against attacks on multiple levels and against many high-value targets. It makes extensive use of .htaccess protections so it would only be useful in an Apache environment with mod_rewrite support of course. So in terms of optimization, you optimize your overall security capabilities of your WordPress installation and reduce the attack vectors and chance of attack.
Conclusion
As with most recommendations, your mileage may vary with the plugins recommended here and I encourage you to test several plugins in the same class to find one you feel comfortable with. It has been my experience however that these plugins are the best of their class and meet not only my needs and expectations but also those of my clients and friends whom I’ve shared these with in the past.