Technique #1

This function is useful when you need to display content, excerpt, custom fields, or anything related to the post beyond it's link and title. If you just need a list of linked titles, see the next technique. Put the following function in functions.php

function recent_posts($no_posts = 10, $excerpts = true) {

   global $wpdb;

   $request = "SELECT ID, post_title, post_excerpt FROM $wpdb->posts WHERE post_status = 'publish' AND post_type='post' ORDER BY post_date DESC LIMIT $no_posts";

   $posts = $wpdb->get_results($request);

   if($posts) {

               foreach ($posts as $posts) {
                       $post_title = stripslashes($posts->post_title);
                       $permalink = get_permalink($posts->ID);

                       $output .= '<li><h2><a href="' . $permalink . '" rel="bookmark" title="Permanent Link: ' . htmlspecialchars($post_title, ENT_COMPAT) . '">' . htmlspecialchars($post_title) . '</a></h2>';

                       if($excerpts) {
                               $output.= '<br />' . stripslashes($posts->post_excerpt);
                       }

                       $output .= '</li>';
               }

       } else {
               $output .= '<li>No posts found</li>';
       }

   echo $output;
}

Usage

After you've made the function. Put the following in the sidebar or wherever you like the recent posts to list..

<?php recent_posts(); ?>

You can give it 2 arguments, the first is the number of posts and the second is whether or not you want to display the excerpts. so recent_posts(2, false) will display the 2 most recent post titles.

Technique #2

<?php wp_get_archives( array(

    'type'            => 'postbypost',   // or daily, weekly, monthly, yearly
    'limit'           => 10,   // maximum number shown
    'format'          => 'html',   // or select (dropdown), link, or custom (then need to also pass before and after params for custom tags
    'show_post_count' => false,    // show number of posts per link
    'echo'            => 1     // display results or return array

) ); ?> 

Technique #3

More succinct version of #1, which also includes a more standardized query string.

<?php
   $recentposts = get_posts('numberposts=12&category=4');
   foreach ($recentposts as $post) :
       setup_postdata($post); ?>
       <li><a href="<?php the_permalink() ?>"><?php the_title(); ?></a></li>
<?php endforeach; ?>

Reference URL