UP & Modi: Some in BJP worried