Congress downplays Trivedi's mid-term poll remark