VS 2005 - Odd Occurrence With SQL Connection

I have a an application that opens a SQL connection and over the course of a few hours creates and disposes of tens of thousands of DataReader objects. You could look at it as just one big loop. As the time goes by, the memory usage steadily climbs until the application/system runs out of memory and crashes.

Psuedo code would look something like
SQLConnection.Connect
Do Until x =1000000
Create New DataReader
Use DataReader
Close DataReader
[Code] .....

Even though I dispose of the DataReader object when I am done with it, the memory usage keeps climbing. I found that every time I am finished with the DataReader I have to close the SQLConnection and dispose of it. When I do that it keeps the memory usage very low and the processor usage is barely noticeable. I would think I could just open the SQL connection once and close it when I am done.