Fun stuff to do if you're visiting from out of town (or if you want to be a "tourist in your hometown"). Remember to dress in layers—it can quickly change from warm to cold (especially with the fog).
I also highly recommend the Free Walking Tours run by the San Francisco City Guides—they are run by informative volunteers (they ask for a donation at the end of the tour) sponsored by the public libraries. They have a wide range of tours around the City, from Victorian houses to Chinatown to "Alfred Hitchcock's San Francisco". Another fun way to experience neighborhoods are Stairway Walks (also in this book).
Cable Car Ride - the quintessential San Francisco treat! The best ride is from Powell & Market on the Hyde line to Ghirardelli Square
buy your ticket in advance (or use a Clipper card); be prepared for a long line at Powell & Market especially on weekends...if you have a small group and don't care where you sit/stand, you might be able to squeeze on at later stops (e.g. on Nob Hill)
Lombard Street - near the cable car drop-off at Ghirardelli Square, this "crookedest street" is fun to traverse by foot (or car, if you're willing to wait)
Alcatraz - this island prison is now a National Park; the ferry ride alone is worth the price of admission, and the audio tour of the prison is fascinating. Book early at alcatrazcruises.com (other websites gouge you with more expensive tickets)...during winter the night tour is also fun.
Coit Tower - this monument at the top of Telegraph Hill has outstanding views, Depression-era murals, and nearby stairs leading to the waterfront
Pier 39 / Fisherman's Wharf / Ghirardelli Square - these are all tourist traps, basically outdoor malls. However, you can still have fun there—many people enjoy watching the sea lions at Fisherman's Wharf. (There are even old school local places like the Eagle Cafe tucked away in Pier 39...now called the Tin Fish? Another good lunch spot is Boudin's bakery/cafe, where you can try the iconic clam chowder in a sourdough bread bowl, or check out the demonstration bakery. The Musée Mécanique is also a fun collection of old-school gaming machines. The nearby Maritime Park also has some historic sailing ships and a World War II submarine.)
Embarcadero Waterfront - a nice pedestrian/bike area along the water; it can be cold/windy but when the fog is out, it has beautiful views of the bay.
Ferry Building - a destination for foodies (e.g. try Acme bread with Cowgirl Creamery cheese) and, as the name implies, you can catch a ferry to various places in the Bay Area (see "Ferry Ride", below)
Exploratorium - a very kid-friendly, hands-on science museum (that's also great for adults)
SF MOMA - the San Francisco Museum of Modern Art has interesting exhibits in a building just south of Market street, near Moscone Convention Center
Asian Art Museum - near San Francisco's City Hall, this museum has an interesting collection of art from all over Asia
SF Giants at AT&T Park - if the San Francisco Giants are playing, you can check out a baseball game (cheap tickets are often available last minute on www.stubhub.com)...if not, you can go for a stroll along the outside of the park, or even take a tour.
Golden Gate Bridge - see this beautiful span up-close and personal, and walk across (or bike--see "Ferry Ride", below)
Presidio - near the Golden Gate Bridge, this former Army base (originally founded by the Spanish) features the Crissy Field walkway, a museum about Walt Disney, and the offices of Lucasarts (complete with Yoda fountain)
Golden Gate Park - home to great museums (such as the California Academy of Sciences natural history museum, and the DeYoung art museum) and other wonders (such as bison, waterfalls,paddleboats, frisbee golf, windmills, and botanical gardens), this park bifurcates the western "Avenues" of the City.
Legion of Honor - perched in a beautiful location overlooking the Golden Gate Bridge, this museum features mostly older Western art
Ocean Beach - it's cold and windy (and often foggy), so don't expect a Baywatch-style beach...but there are waves and sand. Well-wetsuited surfers enjoy the waves, and on a nice day it's still gorgeous. The nearby Sutro Heights and Land's End parks all have great views when the fog's out (Land's End is a popular, easy hike that overlooks the Golden Gate Bridge).
Hidden SF's Guide to SF Neighborhoods has great information...some of Luke's specific favorites are below:
Chinatown - the oldest part of San Francisco, this neighborhood still has a sizable Chinese population. Check out the Golden Gate Fortune Cookie Factory (fortune cookies were invented in SF), the Tin How Temple, the touristy but fun Grant Ave, and the bustling markets of Stockton St. Golden Gate Bakery also has great egg custard tarts (dàn tà), although they are inconsistently open. Lots of good info on Hidden SF .
North Beach - just north of Chinatown is this historically Italian neighborhood, which still has a handful of Italian establishments (including the best Naples-style pizza in the city). My favorite SF gelato is the next neighborhood over, though (Gio Gelati in the Marina District). Not far from Coit Tower/Telegraph Hill (see above). Beat poetry fans will want to check out City Lights Bookstore, former haunt of Allen Ginsberg and Jack Kerouac. More ideas at Hidden SF.
Mission District - traditionally a Latino (Latin American) neighborhood, it has recently gentrified, leading to an interesting mix of hipster cool and Latin spice...some of the trendiest restaurants and eateries are here (don't miss Tartine Bakery, for instance). Valencia is cool for window-shopping, including 826 Valencia's Pirate Store. 24th Street has cool murals and more Latino flavor (e.g. at La Palma, which makes fresh tortillas; some of the best murals are on Balmy Alley). On a nice day, hang out and enjoy the view at Dolores Park. Also check out Precita Eye's weekend tours of Mission murals. Also check out Dandelion Chocolate to watch them make chocolate & get delicious chocolate stuff (they run free tours M-F at 6 PM and Sat at 11 AM, but are typically booked *way* in advance, especially for Saturday)
Castro - since the 60's this has been a gay mecca, although heteros are also welcome. The Castro Theatre often has cool old movies (with an organ) and sing-a-longs.
Haight-Ashbury - the historical home of hippies, this neighborhood still has a counter-culture vibe
Pacific Heights, Nob Hill, Russian Hill - ritzy, nice places to live but probably not much of interest to a tourist...Nob Hill does have fancy hotels if you're into that (e.g. the Mark Hopkins has a classic "old- school" bar, The Top of the Mark, on its top floor, with stunning views)
Hayes Valley, Noe Valley, Cole Valley, Bernal Heights, Glen Park, Potrero Hill, Dogpatch - nice neighborhoods with cool shops and restaurants (e.g. liquid nitrogen ice cream at Smitten in Hayes Valley, which now has branches elsewhere), but probably not of interest to tourists
However, on a clear day there are great views from Bernal Heights Park and from Twin Peaks
Richmond/Sunset - the aforementioned "avenues", north and south of Golden Gate Park, respectively, are close to the ocean (and thus often foggy), with lots of relatively inexpensive ethnic restaurants and shops
Skyline Ridge - this ridge, along the Santa Cruz Mountains, allows you to see both the Ocean and the Bay on a clear day (trails not far from here)
Computer History Museum - in Mountain View, includes an original Cray-1 and various other cool computer history
Beaches / Highway 1 - Highway 1 follows the coast from San Francisco all the way down the California Coast. Some parts of it have breathtaking views of the Pacific Ocean, and various public beaches dot the coast along the way, including a popular one in Pacifica (just south of SF). Stop by Duarte's Tavern in Pescadero or Dad's Luncheonette in Half Moon Bay along the way.
Stanford University - it's interesting to take a tour of this sprawling campus. Hoover Tower has great views from the top. The Stanford Cantor Arts Center features various art, Rodin sculptures, and the "golden spike" used to connect the first trans-continental railroad. Memorial Church is also worth a peek. You can also hike or jog in the Stanford foothills, near "the Dish", offering a great view of the campus and Silicon Valley. There are also tours of the Stanford Linear Accelerator Center the 1st and 3rd Friday of the month at 1:30 and 3:30 PM.
NASA Ames - at Moffet Field, just south of the Google campus, has an exploration center you can visit.
Rosicrucian Egyptian Museum - of interest to people who like Egypt, or who want to learn about the odd Rosicrucian Order
Winchester Mystery House - a bit of a tourist trap, but interesting/spooky
Tech Museum of Innovation - in downtown San Jose
Hiking - Bay Area Hiker has a great catalog of hikes all around the region, including a list of favorites
Ferry Ride - On a nice day, taking a ferry ride is a lot of fun (many depart from the Ferry Building). You can go to Angel Island (the "Ellis Island of the West"), Tiburon (tony place with gorgeous views), Sausalito (cute but touristy town just north of the Golden Gate Bridge, Hidden SF has more), etc. One fun activity is biking across the Golden Gate Bridge, then taking the ferry back from Sausalito or Tiburon (warning: the bridge itself is usually very windy).
Marin County - beyond just Sausalito and Tiburon (see "Ferry Ride", above), Marin has a lot of great open spaces; the most famous are the Marin Headlands (more info at GGNP Conservancy and Hidden SF sites; it's also accessible from SF on weekends via public Muni bus 76X!), Point Reyes, Mount Tamalpais ("Mount Tam") and Muir Woods. Foodies might want to venture up to Point Reyes Station (home of Cowgirl Creamery) and Tomales Bay (where you can buy fresh oysters from Hog Island or Tomales Bay Oyster Companies); Point Reyes National Seashore also has some great hikes
East Bay - across the Bay Bridge, or under the Bay (using BART), is the city of Oakland (including Jack London Square, a nice waterfront area, and Lake Merritt) and a number of other interesting towns (including Berkeley, home of UC Berkeley). Lots of interesting stuff here: For example, "California cuisine" got its start at Chez Panisse, a restaurant in Berkeley. The weather is usually warmer / sunnier than SF, so it offers good outdoor/hiking activities, too.
Monterey County - south of San Francisco, this area features the famous Cannery Row, the Monterey Bay Aquarium, and the cute town of Carmel-by-the-Sea. See the Monterey page for more.
Wine Country - Some of the US's most famous wines are from just north of San Francisco in the Napa and Sonoma valleys; see the Wine Country page for more.
Yosemite - this stunning national park is a 5+ hour drive away. Plan ahead, since camping/lodging is often in short supply (especially in the summer).