Infighting worries BJP chief