BG meters and margins of error - any thoughts

[m1dnc]

Note: This thread was split from an earlier discussion here: http://www.diabetessupport.co.uk/boards/showthread.php?t=29438 to avoid confusing the original thread.

Meters have a 20% error margin so these could technically be the same reading.

Sorry to be nit-picking, but my understanding of the 20% error margin for meter systems is that a meter might give a result on any particular sample that is up to 20% away from the "true" value of that sample, but using the same meter with test strips from the same batch the level of imprecision is much, much lower between tests.

I've just checked the patient leaflet with my test strips (Accu-Chek Aviva) and this gives the reproducibility (i.e. day-to-day imprecision) as less than 1.9%.

So there is a meaningful difference between a reading of 7.2 and a later one of 8.2, when carried out using the same meter and test strips from the same batch. If this were not the case the basal tests that the pumpers are required to do where we are trying to keep our results within a fairly tight margin over an extended period, would be pretty pointless.

That being said, I agree with all the comments that a rise from 7.2 to 8.2 overnight is not at all unusual, and not something I personally would get too concerned about.

 

[everydayupsanddowns]

I've just checked the patient leaflet with my test strips (Accu-Chek Aviva) and this gives the reproducibility (i.e. day-to-day imprecision) as less than 1.9%.

So there is a meaningful difference between a reading of 7.2 and a later one of 8.2, when carried out using the same meter and test strips from the same batch. If this were not the case the basal tests that the pumpers are required to do where we are trying to keep our results within a fairly tight margin over an extended period, would be pretty pointless.

I've spoken to a number of Pharma companies on this very issue, and while mostly what you suggest is probably true, the ISO guidelines +/-20% from a lab test only apply 95% of the time. 5 tests out of a hundred could be outside of that. Apparently the manufacturing process cannot guarantee exactly the same amount of enzyme/reactive agents are sprayed/absorbed onto/remain active on the strips, even within the same pot. I don't know why this is given the level of manufacturing precision available in other areas, but I have been told it more than once by different companies.

For my own part I believe the +/- 1.7mmol/L 'window' often used for a basal test is at least in part designed to allow for strip-to-strip variability. Certainly anything within, say, 2mmol/L being considered 'pretty much the same'.

You only have to test your own fingers a few moments apart to see some sort of variation. And of course I suppose blood itself will not be entirely homogenous and the blood in your feet might have a slightly different makeup to that in your fingertips.

I'd love more accuracy, but at least we can cope with that we've got for the most part - and the PIL recommendations always caveat BG readings with "if you don;t feel like the number you got, then retest" statements!

 

[m1dnc]

I've spoken to a number of Pharma companies on this very issue, and while mostly what you suggest is probably true, the ISO guidelines +/-20% from a lab test only apply 95% of the time. 5 tests out of a hundred could be outside of that. Apparently the manufacturing process cannot guarantee exactly the same amount of enzyme/reactive agents are sprayed/absorbed onto/remain active on the strips, even within the same pot. I don't know why this is given the level of manufacturing precision available in other areas, but I have been told it more than once by different companies.

For my own part I believe the +/- 1.7mmol/L 'window' often used for a basal test is at least in part designed to allow for strip-to-strip variability. Certainly anything within, say, 2mmol/L being considered 'pretty much the same'.

You only have to test your own fingers a few moments apart to see some sort of variation. And of course I suppose blood itself will not be entirely homogenous and the blood in your feet might have a slightly different makeup to that in your fingertips.

I'd love more accuracy, but at least we can cope with that we've got for the most part - and the PIL recommendations always caveat BG readings with "if you don;t feel like the number you got, then retest" statements!

Mike. My earlier post was in reply to DeusXM's bald comment that "meters have a 20% error margin". As you know, the issue of "errors" in scientific measurement is not quite as straightforward as might first appear and really conflates a number of different sources of lack of confidence in the results that we record. Volumes have been written about this and we could easily go completely off-topic very quickly.

Leaving aside for a moment the issue of hypos, where we are looking for absolute accuracy (i.e. is my bs above or below 4 because I have to decide whether to take extra carb?), I would argue that us practicing diabetics should be most interested in the reproducibility of the results obtained from our meters. Thus, if the "true" blood sugar of my finger-prick sample is 6.0, and my meter records a value of 5.0 (or 7.0 for that matter, i.e. within a 20% margin of error) I don't really care so long as it tells me 5.0 (or 7.0) or thereabouts every time I test when the "true" value is 6.0. Of course it's the 'thereabouts' that really matters and that was what I was trying to address in my earlier post and which was at the heart of the op and DeusXM's comment.

The Accu-Chek Aviva stuffer states:

"Reproducibility(day-to-day imprecision): The mean imprecision is <1.9%. In a typical series of tests, a coefficient of variation of 1.8% was obtained."

Like you, I have tried the experiment of repeat testing within a few minutes and found small variations, but not usually more than plus or minus 0.1 or 0.2 or so, which is consistent with the reproducibility range quoted by Roche.

I haven't had the benefit of looking at ISO 15197, which I understand is the relevant standard, because the robbing b's at the ISO want to charge me 134 Swiss Francs to download a copy 😡, but is the 95% figure that you mention referring to the number of outliers that would be tolerated? In the Accu-Chek stuffer, Roche refer to "System accuracy according to ISO 15197: 198 out of 200 samples (99.0%) are within the minimum acceptable performance criteria." I've always thought that the possibility of an outlier was behind the caveat to ignore a meter result if you really feel it's bonkers, although I would re-test or try to find a reason why it doesn't make sense (like I've just dipped my finger in a bowl of sugar before I tested).

This is a very interesting topic. Perhaps you could send me a PM if you want to discuss it further, so that we don't bore everyone else.

 

[robofski]

This is a very interesting topic. Perhaps you could send me a PM if you want to discuss it further, so that we don't bore everyone else.

This is an interesting topic and you're not boring me please don't take you're debate private! Maybe a new thread "Tester Error Margins...Discuss" would prevent this thread from getting too complex! 🙂

 

[everydayupsanddowns]

I didn't really have much of an axe to grind about meter accuracy until fairly recently (I'd only ever used one brand and had got used to the general 'style' of results it provided).

However I was given a chance to trial a new meter to the UK market, and just happened to run some comparitive tests. The results of which pretty much decided me that I coulld not use that meter (even thought there was much about it I really liked).

You can read more on that here:
http://www.everydayupsanddowns.co.uk/2012/03/ibgstar-review-blood-glucose-meter-for.html
and Sanofi's response here:
http://www.everydayupsanddowns.co.uk/2012/03/ibgstar-designed-to-read-high.html


This topic seems to get a lot of attention in the diabetic community.

More thoughts here: http://sixuntilme.com/blog2/2010/03/accuracy_in_glucose_meters.html
here: http://lifeafterdx.blogspot.co.uk/2012/04/four-percent-of-time-your-test-strips.html
and here: http://portablepancreasgirl.com/2012/04/04/too-much-to-ask/

 

[m1dnc]

Mike. Thanks for all these links. I'll spend some time looking through these. At first glance, the most interesting (apart from yours, of course!) seems to be the link through to the FDA conference in 2010 on meter accuracy. There's a presentation there from a Dr Mitchell Scott "Analytical Performance of Blood Glucose Meters: State of the Art" that looks well worth careful scrutiny.

 

[everydayupsanddowns]

I was reminded yesterday of a couple of small-scale research papers I'd read a while ago concerning/comparing Freestyle and OneTouch meters for accuracy. General expectations of 5-6% off lab tests most of the time if I remember right.

 

[m1dnc]

OK. I've spent many happy hours reading the transcript of the 2-day meeting in 2010 organised by the FDA to review the current standards for blood glucose meters, and particularly their accuracy, precision and susceptibility to interference (e.g. from administration of medicaments other than insulin etc.).

Here are some points that I noted, in no particular order. I'll just add the usual caveat that this is my personal interpretation of the matters discussed and you should not make any clinical decision based on these comments without first discussing with your healthcare professionals.

Firstly, the background to the meeting. The current international standard for meters is ISO 15197. Amongst other things, this states that the minimum system accuracy for meters/strips requires them to be accurate to + or - 0.83 for readings <4.2, and to + or - 20% for readings > or = to 4.2. Both these definitions are to 95% confidence limits. The system in this context is the combination of meter and strips meant for it.

This is the minimum standard, but in reality most modern meter systems (as of 2010) met these requirements without difficulty and most were accurate to + or - 15%, and some were even better than that.

As far as precision is concerned (i.e. repeatability and reproducibilty), most meter systems were better than 5% when properly operated.

The accuracy/precision of the meter system should be seen in the context of all the errors that can arise in patient treatment. In particular, other sources of error include, with their probable range of inaccuracy:

Carb counting: 15-20%
Carb/insulin ratio: 10-25%
Administration variability (i.e. the accuracy of syringe/pen/pump to delivery the desired dose): up to 20%
Insulin absorption variabilty once administered: depends on insulin but e.g. for Humalog, 20%

From these, the total error for the whole treatment process can be estimated at about 27% of which only about 1-3% can be put down to the meter system errors.

The biggest source of error in home testing is inadequate hand washing. Four out of five patients do not wash their hands properly before testing.

Outliers (i.e. completely random and inexplicable results) are potentially a problem, especially for making critical decisions for treating/not treating hypos, since the patient has no idea whether the result is an outlier. Around 0.1% (i.e. 1 in a 1000, or once or twice a year if you're testing 4/5 times a day) of results are estimated to be outliers.

The current 20% accuracy standard means that up to 10% of hypos could be missed. If the standard was tightened to 15% (which most current meter systems meet anyway), that would go down to 5%. At 10%, it would drop to 1%, and at 5% hypos would always be detected. The best meters (at 2010) achieve 5% accuracy about 70% of the time.

Meter systems could probably be engineered to get better accuracy but, as in all things, it's a balance of priorities with other factors like convenience (blood drop size, measurement time, size etc) and cost.

Most patients rarely test their meters with the control solutions supplied by manufacturers. This is something that patients should not neglect.

Always use in-date strips. The strips can be affected by heat and humidity. Never leave the strip container open, nor in a car in sunny weather.

There was a lot of stuff about meter system accuracy when they are used in critical care situations in hospitals (e.g. concern about relying on consumer meters for testing unconscious patients), and some interesting discussion about using consumer meters to calibrate cgm systems.

My own view after reading all this stuff, is that I have little option but to accept the reading that my meter gives at face value and make my treatment choices accordingly, but always with the thought at the back of my mind that the reading could be out - re-test if I think there's something funny going on, and dig out that bottle of control solution. I'm also going to be more careful to make sure I wash my hands every time I test!

 

[everydayupsanddowns]

An excellent review m1dnc thank you very much for doing that 🙂

Pretty much backs up what I've felt for some time.

I am *terrible* for not washing hands though [smacks wrist]

The scariest stats in your post were the levels on inaccuracy of all the things like dose administration and absorption. The possibility of up to +/-20% of all of those wading in at once and it's no wonder we think there's a malevolent D fairy messing things up!

 

[robofski]

Good review.

Off to wash my hands 😉

 

[Northerner]

What an excellent review and all good to know, thank you for that 🙂 So, we are supposed to wash our hands before testing? 😱 😉

 

[Vicsetter]

What an excellent review and all good to know, thank you for that 🙂 So, we are supposed to wash our hands before testing? 😱 😉

And DRY them, don't want to dilute the sample do you.

My experience has been, and I thought the meter manufacturers test results bore this out, that meters were more accurate close to 4.0mmol. Certainly if meters were 20% out over the whole range then a reading of 4 could be a hypo or not, but you should know which. a reading of 3 is definitely a hypo. So the only problem (sorry you T1s) is if you are adjusting your injections to your meter reading. In my case I will increase my morning jab to 80 units if my fasting bg is >8, if it's less than 7 then I will stick to 60. Those are big numbers and it's a very course adjustment which will be adjusted during the day by adjusting my food intake, and it's not health critical (I don't think).

As a slightly different but related thought, does anyone know what accuracy the insulin is made to? I sometimes have had days with unexplained reading which I have related to the insulin not working as it usually does (is this possible?)

 

[trophywench]

Not so much the insulin Vicsetter, in my experience - usually the variable absoption rate of different bits of my body - I have a huge dent on one thigh where the fat's disappeared completely and there are several bits of my tummy/midriff which are now rubbish, and short of delineating then with an indelible marker or getting a dotted line round em tattooed, it's quite difficult to avoid them - I mean you hardly get the steel tape measure out when you jab, do you? and I spose a protractor to measure how far and at what angle they are from my navel!

Rarely, it's been 'off' insulin. So that's twice in the last 40 years for me - once with porcine Ultralente and the other with Humulin S. Both went chrystaline pfftt - just like that - but I had thought the previous day 'Is there summat up with this insulin, I'm a bit high?'

 

[m1dnc]

The scariest stats in your post were the levels on inaccuracy of all the things like dose administration and absorption. The possibility of up to +/-20% of all of those wading in at once and it's no wonder we think there's a malevolent D fairy messing things up!

Yes, that shocked me too. But if you think about it, at any one time some of those things will work one way and some the other, so might tend to cancel each other out. I guess that's why the overall level of inaccuracy comes out at about 27%.

I thought the meter manufacturers test results bore this out, that meters were more accurate close to 4.0mmol. Certainly if meters were 20% out over the whole range then a reading of 4 could be a hypo or not, but you should know which. a reading of 3 is definitely a hypo.

That may or may not be true, but it is not reflected in the ISO standard. That states that the meter system has to be accurate to + or - 0.83 units for readings below 4.2 i.e. this is not expressed in percentage terms. At 4.2 this equates to 20%, but for lower readings the allowed error becomes progressively greater percentage-wise. In practice it was reported that modern meter systems are more accurate, as you say. My Accu-Chek Aviva leaflet states that it is within + or - 0.56 units for 100% of results <4.2, and within + or - 0.28 units for more than 80% of results <4.2.

As to insulin manufacturing standards, I have not looked into this. But note that in my earlier post I referred to the variability in insulin absorption once administered. In the case of Humalog, for example, this was quoted to be up to 20%.