CPM leader refuses to quit