Delhi has a large number of hostels, many of them centered around the tourist/backpacker hubs.
Chandni Chowk:
Chandni Chowk originally meaning moonlit square or market, is one of the oldest and busiest markets in central north Delhi, India.Chandni Chowk is the major street in the walled city of Old Delhi, which was originally called Shah Jahanabad. The walled city which includes the Lal Qilla Red Fort of Delhi was established in 1650 AD, by the Mughal Emperor, Shah Jahan.
Paharganj:
This street, also referred to as Main Bazaar, is opposite New Delhi railway station and has many cheap hotels. It's noisy, filthy and full of touts, but it's also cheap and central and thus very popular with budget travelers. A double room with attached bathroom is 200-300 rupees (or less). Note that the Delhi Metro exits are on the Ajmeri Gate side of the New Delhi Railway station, so you'll need to cross over the railway station (Platform Ticket is not needed for entering the station, see the above article on train station cons) to go to Paharganj.
Majnu ka Tilla:
Majnu ka Tilla is a compact Tibetan settlement and the place of departure and arrival for buses to/from Dharamsala, the home of the Tibetan Government in Exile and the Dalai Lama. Stay here if you have an interest in Tibetan culture, politics and religion, or if you need something quieter (and just slightly more expensive) than Paharganj.
An auto-rickshaw from New Delhi train station should cost around 50 rupees (use the prepaid stand). The Vidhan Sabha metro station is also nearby and popular. From there cycle-rickshaws charge 15 rupees and take about five minutes.