gurrier makes a good point. It may be that the lactose was exacerbating your condition, but not necessarily causing it. Taking a break and re-introducing in small amounts seems like a reasonable next step, just to see.