Table of Contents

Introduction and Background 3

Methods of Data Collection 4 Results and Interpretations Descriptive Statistics 5
A. Stem and Leaf Plot 7
B. Box Plot 8
C. Histogram 9
D. Probability 11
E. 95% Confidence Interval for Mean 13
F. Scatter plot 14
G. Simple Linear Regression 15
H. Multiple Linear regression 16
I. Residual plot 17
J. Mean T-test 18
Summary and Conclusions 19

Introduction and Background

In medical device manufacturing there are a lot of proteins that are produced such as antibodies, which require concentration determination. These protein concentration determinations necessary for downstream manufacturing processes to make sure the correct protein concentrations are used to meet product specifications and product performance. In the testing areas there are several test methods used by the laboratory to determine protein concentration. One method is the Amino Acid Analysis method (AAA). This method takes the protein and “unzips” the DNA strand of the protein and analyzes the protein for the different Amino Acids, then calculates a theoretical protein concentration based on the testing. Another method employed is the UV absorbance method (A280), where UV light (wavelength 280) is passed through the sample and hits the detector on the other side of the sample, at which point a measurement is made. The instrument will then use the light reading to determine a protein concentration of the material. Protein concentration readings from both instruments are recorded as mg/ml.
In this analysis we want to demonstrate that the currently used method of AAA, can be replaced with the A280 method without any impact to protein concentration determination. The instrumentation for the AAA method is old and obsolete, it is very costly and time consuming to execute. The laboratory wants to demonstrate that switching to A280 will produce the same results at a much cheaper cost.

Methods of Data Collection

All of the data collected was collected from previous testing and supplemental testing performed to support the change. Each lot of protein produced was tested using test methods, AAA, and A280. Manufacturing targets a concentration range of
0.6- 0.7 mg/ml.

Results and Interpretations

Descriptive statistics

Shown below is the Descriptive Statistics for the AAA and A280 test methods for protein determination. Based on this analysis there was 28 samples analyzed, the average concentration using AAA was 0.732 compared with the average A280 concentration of 0.7304. The standard deviation was 0.077 (AAA) and 0.0716 (A280). The Empirical Rule states that within 3 Standard deviations from our mean will capture 99.7% of all concentration values. The interquartile range will capture the middle 50% of the concentration values for both test methods.

Descriptive statistics Descriptive statistics mg/ml AAA mg/ml A280. count 28 count 28 mean 0.732 mean 0.7304 sample variance 0.006 sample variance 0.0051 sample standard deviation 0.077 sample standard deviation 0.0716 minimum 0.6 minimum 0.62 maximum 0.9 maximum 0.9 range 0.3 range 0.28 1st quartile 0.700 1st quartile 0.6775 median 0.700 median 0.7300
3rd quartile 0.800 3rd quartile 0.7725 interquartile range 0.100 interquartile range 0.0950 mode 0.700 mode 0.7000 low extremes 0 low extremes 0 low outliers 0 low outliers 0 high outliers 0 high outliers 0 high extremes 0 high extremes 0

Table below shows the number of hours tested for each lot under the two different test methods. From the analysis we can see that there is a large difference in the mean of the test methods. AAA had a mean of 16.36 hours where as the A280 had a mean of 0.346 hours. The range was also significantly different with AAA -6 and A280- 0.4. The analysis also shows that AAA had outliers on both the high and low end of testing times.

Descriptive statistics Descriptive statistics Hours AAA Hours A280 count 28 count 28 mean 16.36 mean 0.346 sample variance 1.87 sample variance 0.020 sample standard deviation 1.37 sample standard deviation 0.143 minimum 13 minimum 0.2 maximum 19 maximum 0.6 range 6 range 0.4 1st quartile 15.75 1st quartile 0.200 median 17.00 median 0.300
3rd quartile 17.00 3rd quartile 0.500 interquartile range 1.25 interquartile range 0.300 mode 17.00 mode 0.200 low extremes 0 low extremes 0 low outliers 1 low outliers 0 high outliers 1 high outliers 0 high extremes 0 high extremes 0

Stem and Leaf AAA/A280

A stem and leaf Plot was performed to see the distribution of the AAA values. The highest frequency of AAA testing values falling in the 0.7-0.79 range. The plot showed a normal distribution with slight skew towards the right.
Stem and Leaf plot for mg/ml AAA stem unit = 0.1 leaf unit = 0.01 Frequency Stem Leaf
4 6 0 0 0 0
12 7 0 0 0 0 0 0 0 0 0 0 0 0
11 8 0 0 0 0 0 0 0 0 0 0 0
1 9 0
28

The steam and leaf plot for A280 values had a normal distribution, slightly skewed to the right. The highest frequency of values fell in the 0.7-0.79 range.

Stem and Leaf plot for mg/ml A280. stem unit = 0.1 leaf unit = 0.01 Frequency Stem Leaf
9 6 2 3 3 6 6 6 7 8 8
13 7 0 0 0 0 3 3 4 5 5 6 6 7 8
5 8 0 0 0 1 8
1 9 0
28

Box Plot AAA

A box plot was performed using the AAA values shows the spread in AAA readings on the instrument. The lowest value in the testing was 0.6 and highest value was 0.9. The IQR or Interquartile range is 0.100, which represents the middle 50% of the values in the sampling. The 1st quartile was 0.700, this is the low end of the IQR and the 3rd quartile was 0.800 representing the high end of the IQR. The box plot shows that the middle 50% of the values fall between 0.7 -0.8, which is on the high side of the target specification for manufacturing; and additional with points out as far as 0.6 and 0.9. The median value in this box plot was 0.7, representing the middle score in the data set. An additional box plot of the A280 values was performed with the above results. The lowest value obtained was 0.62 mg/ml and the highest value was 0.9 mg/ml. The 1st quartile was 0.6775 and the 3rd quartile was 0.775, giving an IQR of 0.0950. In the box plot the middle 50% of the values fall between 0.6775 and 0.775, this is closer to the target concentration specification of 0.6-0.7 mg/ml concentration.

Histograms

Frequency Distribution - Quantitative Hours AAA cumulative lower upper midpoint width frequency percent frequency percent 13 < 14 14 1 1 3.6 1 3.6 14 < 15 15 1 2 7.1 3 10.7 15 < 16 16 1 4 14.3 7 25.0 16 < 17 17 1 5 17.9 12 42.9 17 < 18 18 1 12 42.9 24 85.7 18 < 19 19 1 3 10.7 27 96.4 19 < 20 19 1 1 3.6 28 100.0 28 100.0

From the histogram for hours of time required for AAA testing we observe a left skewed distribution. The histogram also shows a large percentage of the hours falling in the 17-18 range.

Frequency Distribution - Quantitative Hours A280 cumulative lower upper midpoint width frequency percent frequency percent 0.2 < 0.2 0.2 0.1 11 39.3 11 39.3 0.3 < 0.3 0.3 0.1 0 0.0 11 39.3 0.3 < 0.3 0.3 0.1 4 14.3 15 53.6 0.4 < 0.4 0.4 0.1 0 0.0 15 53.6 0.4 < 0.4 0.4 0.1 4 14.3 19 67.9 0.5 < 0.5 0.5 0.1 0 0.0 19 67.9 0.5 < 0.5 0.5 0.1 7 25.0 26 92.9 0.6 < 0.6 0.6 0.0 0 0.0 26 92.9 0.6 < 0.6 0.6 0.0 2 7.1 28 100.0 28 100.0

Analysis of the histogram for the hours A280 shows a bimodal distribution of time required to perform testing. Two peaks are observed in the distribution one at 0.2-0.3 range and another peak at the 0.5-0.6 range. This would indicate possible issues were the test method may have some differences depending on the lot tested, and would require further understanding.

Probability

The probability table below is in cases where AAA testing value is used, what is the probability of getting a 0.9 mg/ml concentration value. If manufacturing gets this value it will be required to perform an additional dilution step. From this analysis we can see that this could happen in this interval 1.46% and 98.54%. The distribution graph shows a Z score of 2.18 away from the mean of 0.732.

normal distribution P(lower) P(upper) z X mean std.dev
.9854 .0146 2.18 0.9 0.732 0.077

The probability table below is in cases where A280 testing value is used, what is the probability of getting a 0.9 mg/ml concentration value. From this analysis we can see that this could happen in this interval 1.02% and 98.98%. The distribution graph shows a Z score of 2.32 away from the mean of 0.734.

normal distribution P(lower) P(upper) z X mean std.dev
.9898 .0102 2.32 0.9 0.734 0.0716

Confidence intervals

Confidence interval – mean A280 test value 95% confidence level 0.7304 mean 0.0716 std. dev. 28 n 2.052 t (df = 27) 0.0278 half-width 0.7582 upper confidence limit 0.7026 lower confidence limit

The chart above was used to find the confidence interval in the A280 test values; the chart represents 28 samples and a 95% confidence level. From the output we can conclude that we are 95% confident that the true mean value of A280 values is captured within the interval of (0.7026, 0.7582).
Confidence interval - mean A280 Test Times 95% confidence level 0.3406 mean 0.143 std. dev. 28 n 2.052 t (df = 27) 0.0554 half-width 0.3960 upper confidence limit 0.2852 lower confidence limit

The chart above represents 28 samples from A280 test times, using a 95% confidence interval. From the output we can conclude that we are 95% confident that the true mean test time for A280 is captured in this interval (0.2852, 0.3960). From this data the standard hours for testing should be set using somewhere within this interval.

Scatter plot

For the scatter plot above, we can observe the relationship between the A280 values and the AAA values obtained. There is a positive association between AAA values and A280 values. The equation of the best fit line is y=0.538x + 0.336. From the index of determination of 0.337, means that 33.7% of the variation is explained by this equation.

Simple Regression analysis

Regression Analysis r² 0.337 n 28 r 0.581 k 1 Std. Error 0.064 Dep. Var. AAA Value ANOVA table
Source SS df MS F p-value
Regression 0.0543 1 0.0543 13.23 .0012
Residual 0.1067 26 0.0041
Total 0.1611 27

Regression output confidence interval variables coefficients std. error t (df=26) p-value 95% lower 95% upper
Intercept 0.2744 0.1264 2.170 .0393 0.0145 0.5342 mg/ml A280. 0.6268 0.1723 3.638 .0012 0.2726 0.9809 Predicted values for: AAA Value 95% Confidence Intervals 95% Prediction Intervals mg/ml A280. Predicted lower upper lower upper Leverage
0.9 0.8385 0.7734 0.9035 0.6916 0.9854 0.244
0.7 0.7131 0.6860 0.7402 0.5786 0.8476 0.042

For the simple regression analysis we attempted to see if the values for AAA had a direct correlation with the values of A280 testing. The best fit line for this regression is y=0.6268x + 0.2744. The r value (correlation coefficient) of 0.581 does demonstrate a positive correlation in the values. The regression also produced a low p value of 0.0012, providing more evidence of a correlation. The Index of determination r² is 0.337, which means 33.7% of the variation can be attributed to the AAA value. For prediction intervals, using 0.9 for mg/ml A280, the prediction interval at 95% was (0.6916, 0.9854). Confidence interval for this regression was, we are 95% confident that the true mean A280 mg/ml is captured in this interval (0.2726, 0.9809).

Multiple Regression Analysis
Regression Analysis R² 0.469 Adjusted R² 0.427 n 28 R 0.685 k 2 Std. Error 0.058 Dep. Var. AAA Value ANOVA table
Source SS df MS F p-value
Regression 0.0756 2 0.0378 11.06 .0004
Residual 0.0855 25 0.0034
Total 0.1611 27

Regression output confidence interval variables coefficients std. error t (df=25) p-value 95% lower 95% upper
Intercept 0.2712 0.1154 2.351 .0269 0.0336 0.5088 mg/ml A280. 0.7274 0.1623 4.481 .0001 0.3931 1.0617
Hours A280 -0.2031 0.0814 -2.495 .0196 -0.3708 -0.0354 Predicted values for: AAA Value 95% Confidence Interval 95% Prediction Interval mg/ml A280. Hours A280 Predicted lower upper lower upper Leverage
0.9 0.7 0.7837 0.7090 0.8584 0.6420 0.9254 0.385

The regression above used two variables, mg/ml A280, and hours A280. The equation for this multiple regression is y=-0.2031(hours A280) +0.7274(mg/ml A280) +0.2712. Looking at the global usefulness test, F=11.06 and a p value of 0.0004, indicating a very useful model. The value used to predict AAA result was 0.9 mg/ml (A280) and 0.7 (hours A280). The predicted result of AAA value would be 0.7837. The prediction interval at 95% confidence would be (0.6420, 0.9254).

Residual Plot

In the residual plot above are points are randomly scattered around the line, with no observable pattern. Therefore, this is an acceptable model to use. This means that there is a relationship between the A280 mg/ml value and the AAA mg/ml values. .

Hypothesis testing

The hypothesis is that there is no difference in values obtained via AAA and values obtained via A280 testing. To determine if there is a difference or not a T-test was performed on the differences in A280 and AAA values. The A280 testing values were subtracted from the AAA values, this data was then used to perform T test using a 95% confidence interval. The hypothesis is that there is no difference between the values.

Ho: u is equal to 0
Ha: u is not equal to 0
Hypothesis Test: Mean vs. Hypothesized Value 0.00000 hypothesized value 0.00179 mean AAA-A280 0.06832 std. dev. 0.01291 std. error 28 n 27 df 0.14 t .8910 p-value (two-tailed) -0.02470 confidence interval 95.% lower 0.02828 confidence interval 95.% upper 0.02649 margin of error

From the T test results we observe that the t value is 0.14, which falls in the “Fail to reject Ho” portion of the distribution. In order to reject the Ho the t value must be greater than 1.956 or less than -1.956. In view of the fact that the null hypothesis was not rejected we can state there is no statistical difference between the AAA and A280 values.
The p-value is 0.8910, so there is a 0.8910 probability of getting these sample results if the Ho hypothesis is true.

Summary and Conclusions

The purpose of the analysis was to determine if A280 testing could produce equivalent testing results as AAA. The data collected would justify the change in the testing procedure; replacing a timely and costly procedure with a more cost effective solution.

• Based on Descriptive statistics we observe significant differences in the mean hours for each test method. AAA 16.36 hours and 0.346 hours for A280. By switching from AAA to A280, there would be a significant cost savings in cost due to difference in hours.
• From the Box plots of both test methods the spread in values obtained by each test method is very similar IQR scores: 0.100 (AAA) and 0.095 (A280).
• From the Hours AAA histogram we observe a left skewed distribution, with the highest frequency of testing hours at 17-18. In the hours A280 we observe a bimodal distribution, where further analysis is required to understand why there are two populations of testing times.
• Analysis was performed to determine the probability of 0.9 mg/ml value obtained, which would then require manufacturing to perform an additional processing step. The probability of this occurring using AAA and A280 is 1.46% and 1.02% respectively. Thus, a low probability of occurrence.
• We also able to understand the 95% confidence intervals for A280 values and test times. We concluded that we are 95% confident that the true A280 mean value is captured in the interval of (0.7026 and 0.7582) and the true mean test hour value is captured in the interval of (0.2852, 0.3960).
• The Null hypothesis test was used to determine if there was no difference in protein values obtained using A280 or AAA. From that testing at a 95% confidence level, we were able to obtain a t value of 0.14, which failed to reject the Ho hypothesis.
• From the scatter plot of AAA and A280 test values we observe a positive association in the test values.
• In the linear regression analysis we observed a direct association with the AAA values and the A280 values. The regression analysis provided a low p value indicating again a positive correlation between the two methods.
• In the multiple regression analysis we found in comparing p values that the variable that contributed most was the A280 concentration value, most evidence of a correlation between AAA and A280.

This analysis demonstrated that the use of AAA or A280 test methods has no impact on the concentration value obtained by these test methods. This analysis will allow the lab to switch from a higher cost/time method (AAA) to a lower cost/time method (A280).

Data used for this analysis

Lots AAA Value mg/ml A280. AAA-A280 Hours AAA Hours A280
1 0.7 0.63 0.07 15 0.2
2 0.8 0.75 0.05 16 0.3
3 0.7 0.8 -0.1 17 0.2
4 0.8 0.7 0.1 16 0.3
5 0.7 0.68 0.02 14 0.5
6 0.8 0.74 0.06 17 0.2
7 0.9 0.8 0.1 13 0.2
8 0.6 0.7 -0.1 17 0.4
9 0.8 0.73 0.07 17 0.2
10 0.7 0.7 0 15 0.2
11 0.7 0.66 0.04 18 0.5
12 0.7 0.66 0.04 17 0.3
13 0.8 0.9 -0.1 19 0.4
14 0.8 0.76 0.04 17 0.3
15 0.7 0.62 0.08 15 0.2
16 0.8 0.76 0.04 17 0.2
17 0.7 0.67 0.03 16 0.2
18 0.8 0.78 0.02 16 0.5
19 0.6 0.63 -0.03 17 0.5
20 0.8 0.73 0.07 17 0.4
21 0.7 0.81 -0.11 14 0.5
22 0.7 0.75 -0.05 17 0.6
23 0.6 0.7 -0.1 18 0.5
24 0.7 0.68 0.02 18 0.2
25 0.6 0.66 -0.06 17 0.2
26 0.8 0.8 0 15 0.4
27 0.7 0.77 -0.07 16 0.6
28 0.8 0.88 -0.08 17 0.5 Average of 28 lots 0.73214 0.73036 0.00179
Std Deviation of 28 lots 0.07724 0.07157 0.06832