What’s Wrong With OsteoStrong?: A Critical Analysis and Review

Dr. Giangregario identified seven significant shortcomings in the OsteoStrong study.

1. Failure to Meet Basic Scientific Standards

According to Dr. Giangregorio, the OsteoStrong study fails to meet even the most fundamental standards expected in scientific research:

1. No clear objectives or hypotheses
2. Inadequate information about ethical approval
3. No clear statistical analysis plan describing how results were analyzed
4. Incomplete description of how data was collected and analyzed for many outcomes
5. No clear information about what the control group actually did

Scientific research follows internationally accepted standards and checklists for reporting. Alarmingly, this study is missing most of these essential elements.

2. Lack of Trial Registration

The scientific community agrees that all clinical trial protocols should be registered in a widely available online registry to ensure transparency. This registration number should be included in the published study.

Without a registered protocol, no one can confirm whether the researchers followed their original plan or if they changed their analysis or reporting methods to align with desired outcomes.

Interestingly, there is no information confirming that the OsteoStrong trial was registered. Dr. Giangregorio contacted both the authors and the university’s ethics board to determine if it was registered but received no response.

3. “Low Quality Study” with High Risk of Bias

From what Dr. Giangregorio could determine, this appears to be a very low-quality study with no efforts made to reduce the risk of bias.

Rather than using a randomized controlled trial (the gold standard in research), participants were allowed to choose whether they wanted to be in the control group or the OsteoStrong group. This self-selection creates an immediate bias, as people eager to participate in OsteoStrong may have had different health status or habits than those who opted out.

Additionally, there’s no indication that the study blinded the assessors – meaning the people evaluating outcomes knew which participants were in which group. This knowledge can significantly influence results, whether consciously or unconsciously.

4. Confounding Factors: Osteoporosis Medications

Dr. Giangregorio noted it was unusual that the study enrolled participants who were taking osteoporosis medications. These medications increase bone mineral density on their own, which confounds the results – especially if:

1. The types of medications varied among participants
2. Duration of medication use differed
3. The number of participants was too small to separate medication effects from OsteoStrong effects

The study provides very little information about the types of osteoporosis medications participants were taking, how long they had been on them, or whether they took them consistently. This means the study wasn’t designed to accurately test the effects of OsteoStrong independent of medication effects.

5. The Crucial Question: Does OsteoStrong Improve Bone Density Scores?

For many women considering OsteoStrong, the most important question is whether the study demonstrated that OsteoStrong increases bone mineral density compared to the control group.

Surprisingly, we don’t know the answer because the authors didn’t report any analyses of between-group differences in bone mineral density at the hip or spine. Instead, they reported other statistics that aren’t typically used or accepted in clinical trials, such as:

1. Changes in bone mineral density within each group (but not compared between groups)
2. T-scores
3. The percentage of people who experienced an increase in bone mineral density

The scientific standard is to focus on whether the bone mineral density change in one group is statistically different from the other group. Without this comparison, we cannot draw conclusions about effectiveness.

6. Lack of Proper Statistical Analysis

The study reported changes within each group but did not report statistical analyses comparing between the groups. Without this critical between-group analysis, it’s inappropriate to make conclusions about the effectiveness of OsteoStrong compared to the control.

Dr. Giangregorio points out that the average bone mineral density was already different between groups at the start of the study – something that shouldn’t happen if the groups are properly balanced. There’s also no information about how the researchers handled:

1. Baseline differences between groups
2. Missing data
3. Outliers (unusually large individual changes that can skew averages)

Some participants showed changes that seem implausibly large. Including these outliers can falsely increase the average change reported. Dr. Giangregorio noted several errors and outliers in the paper, suggesting that a closer examination of the data and statistical analysis is warranted.

7. Follow the Money: Who Funded the Study?

Just as we might be skeptical of research funded by pharmaceutical companies, it’s important to note that this study was funded by a private company that owns OsteoStrong franchises.

Although the authors disclosed this funding, there remains significant potential for conflict of interest that could influence how the study was designed, conducted, analyzed, and reported.

8. Scientific Community Response

Dr. Giangregorio believes that the peer reviewers who evaluated this paper before publication may have missed several critical issues. She notes that the paper doesn’t even meet all of the journal’s own authorship guidelines.

The Bone Health and Exercise Science Lab at the University of Waterloo and other scientists globally have made the journal editors aware of their concerns about this paper and have called for its retraction. Several researchers from multiple countries, including researchers at Harvard, have written letters calling for its retraction.

Importantly, the senior author of the study is an editor for the journal where it was published, raising additional questions about the review process.