That's really not what technical debt means, but yeah, you have to maintain tests. There's a cost.
> So what is the optimal number of tests?
NaN. There is no right number of tests. And more test cases does not mean "more coverage."
> while tests cover individual inputs
Depends on the kind of tests you write. When I'm doing property-based tests I like to randomly sample inside equivalence classes to test that I'm setting boundaries correctly. Picked up the habit when I saw that catching bugs when we moved from 32 bit to 64 bit code.
> The fibonacci sequence is well-defined, it has a clear definition, it will never change
Then you don't need automated tests, do you? Manually show that it works once then have automation that diffs that code to make sure the promised "never changing" really happens! Not what I would do, but different strokes for different folks . . .
And I wouldn't do it because the implementation of something non-trivial and not made up as a straw man might have a valid implementation change. Which is a great reason to write behavioral tests for the function.
> calculation()
The proposed tests don't test the interesting stuff. What type can n be? Can the function overflow? Does it slow down when one type is cast to another?
Also, why do you have a function that just returns n * 2 + 6? That's gotta upset those anti-clean-code folks you work with! Ugly high-maintenance tests can point to dev code smells.
> isEligible()
This function does something so trivial that if it were actually used, I bet it's not used everywhere for the check. This is the kind of thing where people use isEligible() in places and just hard-code checks against the age in others. And whenever the age changes, something will break.
Again, the tests seem dumb, but I bet there's a better way to write the production code.
> Is the test simpler than the implementation?
Seriously? If so, that might be a sign that the implementation is too complicated. Functional tests will typically have some kind of setup (possibly complicated) then the simple thing they're checking, in your ideally easy-to-understand function.
> Mocking adds a ton of complexity and reduces the amount of functionality actually being tested. It makes code both harder to test and less valuable to test.
Without knowing more about how you've done mocking and why you say that, I can only sound like one of those zealot snobs when I look down my nose and say "you're doing it wrong." I don't want to do that. So without a solid example here, I can't really respond.
FWIW, I mock to see how the results of different wire calls will affect the code. A small, well-defined set of results. I'd rather do this in unit testing with mocks than stand up the service and surround it with fakes because that's a lot slower and more brittle and I have to think more. And I'd rather stand it up surrounded with fakes than test in a real, full environment because that is slower still, even more brittle, and requires some kind of magical powers to debug.
> Integration tests
> almost by definition you aren't really mocking systems, you're testing the interactions between them that are otherwise hard to test
I bet you have never heard of contract tests. They're a faster, lighter, less brittle, less brain-hurty way of testing interactions between services. I swear I don't get a commission from Pact.io (a SmartBear acquisition), but you should check them out.
> UI tests
> Often change output/behavior intentionally (leading to flaky tests)
That's not what a flaky test is. If you change some code's behavior and don't change the tests and the tests break, they're good tests - fix them!
When a test passes . . . except sometimes it doesn't, so everyone says "run it again," that is flaky. And flakiness sucks. Often caused by not waiting for (or being triggered by, depending whether you're doing polling or push/promises/etc.) the right condition before taking the next action.
> have trivial logic that's easy to verify by looking at it
So, again, I assume you have tests that diff that section of code to verify the tacit promise that the code won't change.
> I don't hate tests (I promise, Avid!), I just try to question them on a case by case basis, and I haven't met a lot of people who do.
I'd suggest trying to understand the tests instead of questioning to prove they're not useful.
> They have a very real organizational cost
Yes. And so does bug fixing. The biggest cost of bug fixing, IMO, is finding the places where there were contradictory expectations. If only someone had coded those expectations as tests . . .
> makes that coverage number go up, so it feels good to add them.
That's a process smell. And I don't just mean the gamification. Code coverage is really important but not for rising numbers. What matters is what's not covered - there be bugs!
That's really not what technical debt means, but yeah, you have to maintain tests. There's a cost.
> So what is the optimal number of tests?
NaN. There is no right number of tests. And more test cases does not mean "more coverage."
> while tests cover individual inputs
Depends on the kind of tests you write. When I'm doing property-based tests I like to randomly sample inside equivalence classes to test that I'm setting boundaries correctly. Picked up the habit when I saw that catching bugs when we moved from 32 bit to 64 bit code.
> The fibonacci sequence is well-defined, it has a clear definition, it will never change
Then you don't need automated tests, do you? Manually show that it works once then have automation that diffs that code to make sure the promised "never changing" really happens! Not what I would do, but different strokes for different folks . . .
And I wouldn't do it because the implementation of something non-trivial and not made up as a straw man might have a valid implementation change. Which is a great reason to write behavioral tests for the function.
> calculation()
The proposed tests don't test the interesting stuff. What type can n be? Can the function overflow? Does it slow down when one type is cast to another?
Also, why do you have a function that just returns n * 2 + 6? That's gotta upset those anti-clean-code folks you work with! Ugly high-maintenance tests can point to dev code smells.
> isEligible()
This function does something so trivial that if it were actually used, I bet it's not used everywhere for the check. This is the kind of thing where people use isEligible() in places and just hard-code checks against the age in others. And whenever the age changes, something will break.
Again, the tests seem dumb, but I bet there's a better way to write the production code.
> Is the test simpler than the implementation?
Seriously? If so, that might be a sign that the implementation is too complicated. Functional tests will typically have some kind of setup (possibly complicated) then the simple thing they're checking, in your ideally easy-to-understand function.
> Mocking adds a ton of complexity and reduces the amount of functionality actually being tested. It makes code both harder to test and less valuable to test.
Without knowing more about how you've done mocking and why you say that, I can only sound like one of those zealot snobs when I look down my nose and say "you're doing it wrong." I don't want to do that. So without a solid example here, I can't really respond.
FWIW, I mock to see how the results of different wire calls will affect the code. A small, well-defined set of results. I'd rather do this in unit testing with mocks than stand up the service and surround it with fakes because that's a lot slower and more brittle and I have to think more. And I'd rather stand it up surrounded with fakes than test in a real, full environment because that is slower still, even more brittle, and requires some kind of magical powers to debug.
> Integration tests > almost by definition you aren't really mocking systems, you're testing the interactions between them that are otherwise hard to test
I bet you have never heard of contract tests. They're a faster, lighter, less brittle, less brain-hurty way of testing interactions between services. I swear I don't get a commission from Pact.io (a SmartBear acquisition), but you should check them out.
> UI tests > Often change output/behavior intentionally (leading to flaky tests)
That's not what a flaky test is. If you change some code's behavior and don't change the tests and the tests break, they're good tests - fix them!
When a test passes . . . except sometimes it doesn't, so everyone says "run it again," that is flaky. And flakiness sucks. Often caused by not waiting for (or being triggered by, depending whether you're doing polling or push/promises/etc.) the right condition before taking the next action.
> have trivial logic that's easy to verify by looking at it
So, again, I assume you have tests that diff that section of code to verify the tacit promise that the code won't change.
> I don't hate tests (I promise, Avid!), I just try to question them on a case by case basis, and I haven't met a lot of people who do.
I'd suggest trying to understand the tests instead of questioning to prove they're not useful.
> They have a very real organizational cost
Yes. And so does bug fixing. The biggest cost of bug fixing, IMO, is finding the places where there were contradictory expectations. If only someone had coded those expectations as tests . . .
> makes that coverage number go up, so it feels good to add them.
That's a process smell. And I don't just mean the gamification. Code coverage is really important but not for rising numbers. What matters is what's not covered - there be bugs!