(No drush!?!? #inconceivable) With all of the fancy ways to upgrade Drupal core floating around: drush, git, diff, patching. What if you only have FTP? As always, the first steps are to a) plan, b) backup, and c) be flexible. It always helps to have a plan. Write it down, think it through, practice on something other than your production site. The basics for updating a Drupal minor version (eg., 6.20 to 6.22) are that you need to save your /sites folder and anything that you've made changes to outside of the /sites folder. Usually those changes might be to .htaccess or robots.txt.
When new security releases come out for Drupal, updating is definitely something that you want to do sooner rather than later. My philosophy has generally been to update my local setup immediately, let all of our developers/users know that I am going to update the shared dev server within two days, then schedule a time to update production based on any issues we find with the dev server. Since we run most of our sites as a rather large multisite, it means updating everybody at once, but hey, that's what dev's for, right? Well, we'll leave that for another discussion.