QuickCheck is an awesome library, so I decided to play with it a bit and also refresh my memory of Haskell.

Intro

To, warm up, let’s remind ourselves how insane floating point arithmetic is. Let’s test if addition of floating point numbers is commutative (x + y = y + x).

$ stack ghci
Prelude> import Test.QuickCheck
Prelude Test.QuickCheck> :{
Prelude Test.QuickCheck| prop_additionIsCommutative :: Double -> Double -> Bool
Prelude Test.QuickCheck| prop_additionIsCommutative x y = x + y == y + x
Prelude Test.QuickCheck| :}

Prelude Test.QuickCheck> quickCheck prop_additionIsCommutative
+++ OK, passed 100 tests.

Ok, so far so good. What about associativity ((x + y) + z == x + (y + z))?

Prelude Test.QuickCheck> :{
Prelude Test.QuickCheck| prop_additionIsAssociative :: Double -> Double -> Double -> Bool
Prelude Test.QuickCheck| prop_additionIsAssociative x y z = (x + y) + z == x + (y + z)
Prelude Test.QuickCheck| :}

Prelude Test.QuickCheck> quickCheck prop_additionIsAssociative
*** Failed! Falsifiable (after 2 tests and 117 shrinks):
2.2204498059835824e-16
3.144251466391179e-2
0.9693770503702299

Nice! Let’s try a few more.

Prelude Test.QuickCheck> quickCheck ((\x y z -> x * y * z == x * (y * z)) :: Double -> Double -> Double -> Bool)
*** Failed! Falsifiable (after 2 tests and 1148 shrinks):
5.0e-324
2.267914075073144
1.1023346291857508

Prelude Test.QuickCheck> quickCheck ((\x y -> x + y - y == x) :: Double -> Double -> Bool)
*** Failed! Falsifiable (after 4 tests and 2097 shrinks):
5.0e-324
4.450147717014403e-308

This is one more reminder not to use double and float types to represent money. If you have X dollars and then someone gives you Y dollars and then changes their mind and takes these Y dollars back, you can end up with 0 dollars in your pocket, even though X was greater than zero. Before taking money from someone, make sure they don’t use doubles for money.

x, y = 5.0e-324, 4.450147717014403e-308;
x > 0           # => true
x + y - y == 0  # => true

Testing Date and Time

Not only floating-pointer numbers are insane. Date and time are extremely weird as well. All these time zones, daylight saving time, leap years, leap seconds. Pure madness. Many of our assumptions about time are wrong.

In order to make my life more complicated, I decided to check whether ActiveSupport::Duration from Rails is associative, but I want to use Haskell’s QuickCheck to do that.

Here are a few instances of the ActiveSupport::Duration class:

1.month    # => 1 month
10.years   # => 10 years
-5.seconds # => -5 seconds

t = Time.zone.now # => Sat, 24 Dec 2016 14:24:41 EST -05:00
5.weeks.since(t)  # => Sat, 28 Jan 2017 14:24:41 EST -05:00

In which case the time to wait is longer: when you first wait D1 and then D2, or when you first wait D2 and then D1? Common sense suggests that it’s the same, but let’s check it.

Let’s get started.

stack new ActiveSupportDates quickcheck-test-framework
cd ActiveSupportDates

There are a few predefined tests in test/Spec.hs, let’s remove them all. Now, testing pure properties is easy. This time, however, we need to communicate with Ruby, thus we need quickcheck with IO. After 15 seconds of searching, we can find this. Looks like this is exactly what we need. Let’s adapt it to our needs.

import Test.Framework (defaultMain, testGroup)
import Test.Framework.Providers.QuickCheck2 (testProperty)

import Test.QuickCheck
import Test.QuickCheck.Monadic

import Data.Char (toLower)
import System.Process (readProcess)

data Duration = Duration TimeUnit Int deriving (Eq, Show)

data TimeUnit
  = Seconds
  | Minutes
  | Hours
  | Days
  | Weeks
  | Months
  | Years deriving (Eq, Show)

instance Arbitrary Duration where
  arbitrary = Duration <$> arbitrary <*> arbitrary

instance Arbitrary TimeUnit where
  arbitrary = oneof
    [ pure Seconds
    , pure Minutes
    , pure Hours
    , pure Days
    , pure Weeks
    , pure Months
    , pure Years
    ]

main :: IO ()
main = quickCheck prop_commutative

prop_commutative :: Duration -> Duration -> Property
prop_commutative op1 op2 = monadicIO $
  run (commutative op1 op2) >>= assert

{-
Builds a Ruby program which applies two given durations to a fixed date
in different order and check whether the output is the same. The program
then is fed to Ruby interpreter. The program prints "true" when the dates
made by applying durations in different order are equal. The output of the
program is converted to Haskell's Bool
-}
commutative :: Duration -> Duration -> IO Bool
commutative op1 op2 = do
  let flags = ["-ractive_support/core_ext/integer/time", "-e", rubyProgram op1 op2]
  output <- readProcess "ruby" flags ""
  return (output == "true")

{-
Example:
`rubyProgram (Duration Seconds 3) (Duration Years 5)` returns a string
equivalent to the following Ruby code:

    tz = ActiveSupport::TimeZone.zones_map.fetch('Eastern Time (US & Canada)')
    t = tz.parse('Fri, 23 Dec 2016 18:30:02 EST -05:00')
    x = 3.seconds.since(5.years.since(t))
    y = 5.years.since(3.seconds.since(t))
    print(x == y)
-}
rubyProgram :: Duration -> Duration -> String
rubyProgram op1 op2 =
    "tz = ActiveSupport::TimeZone.zones_map.fetch('Eastern Time (US & Canada)'); " ++
    "t = tz.parse('Fri, 23 Dec 2016 18:30:02 EST -05:00'); " ++
    "x = " ++ applyDurations op1 op2 ++ "; " ++
    "y = " ++ applyDurations op2 op1 ++ "; " ++
    "print(x == y)"
  where applyDurations x y = toRuby x ++ ".since(" ++ toRuby y ++ ".since(t))"

{-
Examples:
toRuby (Duration Hours 5)
  -- "5.hours"

toRuby (Duration Minutes (-1))
  -- "-1.minutes"
-}
toRuby :: Duration -> String
toRuby (Duration unit amount) = show amount ++ "." ++ map toLower (show unit)

Simpru, right?

Let’s run it:

$ stack test
*** Failed! Assertion failed (after 13 tests):
Duration Days 10
Duration Months 2

And check it in Ruby:

tz = ActiveSupport::TimeZone.zones_map.fetch('Eastern Time (US & Canada)')
t = tz.parse('Fri, 23 Dec 2016 18:30:02 EST -05:00')
# => Fri, 23 Dec 2016 18:30:02 EST -05:00
x = 10.days.since(2.months.since(t))
# => Sun, 05 Mar 2017 18:30:02 EST -05:00
y = 2.months.since(10.days.since(t))
# => Thu, 02 Mar 2017 18:30:02 EST -05:00

Wow, 3 days difference! Not bad. This is because adding the actual duration of adding months depends on the timestamp we are adding these months to.

(t + 2.seconds) - (t + 1.second) == 1.second
# => true
(t + 2.months) - (t + 1.month) == 1.month
# => false

Now, what if we remove months out of the equation? Let’s comment out pure Months from Arbitrary instance definition for Duration and rerun the test.

Failed! Assertion failed (after 50 tests):
Duration Years 17
Duration Weeks (-49)

Let’s have a look.

# => Fri, 23 Dec 2016 18:30:02 EST -05:00
x = 17.years.since(-49.weeks.since(t))
# => Sat, 15 Jan 2033 18:30:02 EST -05:00
y = -49.weeks.since(17.years.since(t))
# => Fri, 14 Jan 2033 18:30:02 EST -05:00

1 day difference. This time, it’s a leap year problem: if we travel from 23 December 2016 to 49 weeks in past, we will get to 15 January (because 2016 is a leap year and 29 February is a thing). Then 17 years to the future - it’s 15 Jan 2033.

On the other hand, if we first go to the future (23 December 2033), and then 49 weeks back, then we’ll get to 14 January (because the 2033 year is not leap and Feb 29 doesn’t exit).

So the problem now is very similar to the previous problem we had with month durations. year durations depend on the year of the date they are applied to. Let’s comment out pure Years and rerun tests.

$ stack test
+++ OK, passed 100 tests.

Hmm, looks promising. But maybe I overlooked something? Let’s increase the number of tests and run it again.

main :: IO ()
main = quickCheckWith stdArgs{maxSuccess = 1000, maxSize=100500} prop_commutative
$ stack test
Failed! Assertion failed (after 10 tests):
Duration Hours 684
Duration Days (-433)

What is this?

x = 684.hours.since(-433.days.since(t))
# => Mon, 16 Nov 2015 01:24:41 EST -05:00
y = -433.days.since(684.hours.since(t))
# => Mon, 16 Nov 2015 02:24:41 EST -05:00

1 hour difference. DST? Most likely. If you run stack test a few times, you’ll see that x and y will be around mid March or mid November. This confirms our suspicion. Well, the problem is the same: 1.day doesn’t actually have a constant length. It can be 23 or 25 hours long, if we jump over DST point.

Let’s exclude days and weeks and rerun tests.

I ran stack test with the new settings multiple times and didn’t get any new failures. It doesn’t mean that the commutative property holds for second/minute/hour durations. It just means that QuickCheck didn’t find any counterexamples.

Conclusion

Current implementation of the test is very inefficient: it starts a process for each single check. My initial implementation was more ugly and a lot faster. Its source code is there. I checked a few more properties there. In that implementation, the Ruby process is started once and it communicates with Haskell via pipe.