SC refuses to direct panel to summon Modi