iio: cros_ec: Fix the maths for gyro scale calculation
authorGwendal Grignou <gwendal@chromium.org>
Wed, 13 Mar 2019 11:40:02 +0000 (12:40 +0100)
committerJonathan Cameron <Jonathan.Cameron@huawei.com>
Sat, 16 Mar 2019 16:08:55 +0000 (16:08 +0000)
Calculation did not use IIO_DEGREE_TO_RAD and implemented a variant to
avoid precision loss as we aim a nano value. The offset added to avoid
rounding error, though, doesn't give us a close result to the expected
value. E.g.

For 1000dps, the result should be:

    (1000 * pi ) / 180 >> 15 ~= 0.000532632218

But with current calculation we get

    $ cat scale
    0.000547890

Fix the calculation by just doing the maths involved for a nano value

   val * pi * 10e12 / (180 * 2^15)

so we get a closer result.

    $ cat scale
    0.000532632

Fixes: c14dca07a31d ("iio: cros_ec_sensors: add ChromeOS EC Contiguous Sensors driver")
Signed-off-by: Gwendal Grignou <gwendal@chromium.org>
Signed-off-by: Enric Balletbo i Serra <enric.balletbo@collabora.com>
Cc: <Stable@vger.kernel.org>
Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
drivers/iio/common/cros_ec_sensors/cros_ec_sensors.c

index 89cb0066a6e0839f49fd68fb2395e8425b174c90..8d76afb87d87c58322b3ee8835ea31f5edc5a834 100644 (file)
@@ -103,9 +103,10 @@ static int cros_ec_sensors_read(struct iio_dev *indio_dev,
                         * Do not use IIO_DEGREE_TO_RAD to avoid precision
                         * loss. Round to the nearest integer.
                         */
-                       *val = div_s64(val64 * 314159 + 9000000ULL, 1000);
-                       *val2 = 18000 << (CROS_EC_SENSOR_BITS - 1);
-                       ret = IIO_VAL_FRACTIONAL;
+                       *val = 0;
+                       *val2 = div_s64(val64 * 3141592653ULL,
+                                       180 << (CROS_EC_SENSOR_BITS - 1));
+                       ret = IIO_VAL_INT_PLUS_NANO;
                        break;
                case MOTIONSENSE_TYPE_MAG:
                        /*