Families of rape accused flee homes in fear