Sql Server - Linq - Limit List To 1 Row Per Unique Values Based On Value (minimum) Of Single Field

I have a stored procedure (I cannot edit) that I am calling via linq.

The stored procedure returns values (more complex but important data below):

Customer Stock Item Date Price Priority Qty
--------------------------------------------------------
CUST1 TAP 01-04-2012 £30 30 1 - 30
CUST1 TAP 05-04-2012 £33 30 1 - 30
CUST1 TAP 01-04-2012 £29 20 31 - 99
CUST1 TAP 01-04-2012 £28 10 1 - 30

I am trying to limit this list to rows which have unique Dates and unique quantities in LINQ. I want to remove items with the HIGHER priority leaving rows with unique dates and qty's.

I have tried several group by's using Max and order by's but have not been able to get a result.

Is there any way to do this via linq?

EDIT:

Managed to convert brad-rem's answer into VB.net.

Syntax below if anyone needs it:

returnlist = (From p In returnlist
Order By p.Qty Ascending, p.Priority
Group By AllGrp = p.Date, p.Qty Into g = Group
Select g.First).ToList