Simulating 2-Sample Confidence Intervals

How well do the 2-sample confidence intervals work with skewed populations?

pop1 = rexp(5000,0.01)
pop2 = rexp(5000,0.02)

hist(pop1,col='gray',main='Population 1 Distribution',xlab='Outcome')

hist(pop2,col='gray',main='Population 2 Distribution',xlab='Outcome')

How well does the 2-sample confidence interval formula for the difference in population means work? Let’s find out.

N = 26
mu1 = mean(pop1)
mu2 = mean(pop2)
mu1

## [1] 99.78321

mu2

## [1] 49.74244

results = c()
for (i in 1:10000) {
  smp1 = sample(pop1,N)
  smp2 = sample(pop2,N)
  test = t.test(smp1,smp2)
  lower = test$conf.int[1]
  upper = test$conf.int[2]
  result = (upper > mu1-mu2) & (lower < mu1-mu2)
  results = c(result,results)
}
table(results)

## results
## FALSE  TRUE 
##   605  9395