Wow, there's a lot of anger in some of these posts.
I've been using Mockito for about 4 years, all in Kotlin. I always found it to be "plenty good" for like 99% of the cases I needed it; and things more complicated or confusing or messy were usually my fault (poor separation of concerns, etc).
I regularly found it quite helpful in both its spy() and mock() functionality.
I never found it meaningfully more or less useful than MockK, though I have heard MockK is the "one that's better for Kotlin". It's mostly just vocabulary changes for me, the user.
I'm going to have to monitor Mockito's future and see if I'll need to swap to MockK at some point if Mockito becomes unmaintained.
"Wow, there's a lot of anger in some of these posts."
If I was OP I'd retire happy knowing that a very thankless job is well done! Given what it does: the more outrage the better. Projects like Mockito call out the lazy and indolent for whom they are and the hissing and spitting in return can simply be laughed at.
10 years this bloke has given his time and effort to help people. He states: nearly a third of his life.
I'll raise a glass and say "was hale" or perhaps wassail as an Englander might.
I'm not sure Kotlin is a hack, but it's growth is directly related to Java not being able to deliver on simple features that would help the day to day lives of developers. So I also can't blame anyone for moving on from the language. I get frustrated with it all the time, especially when Kotlin shows that some feature could work perfectly fine.
Here are some examples that have hit the graveyard: It's been 2 years since exception handling in switch was proposed [0], 3 years since null-restricted types were proposed [1], 4 years since string templates [2], 8 years since concise method bodies [3], and 11 years for JSON parsing [4].
At the end of the day I think there's nothing wrong with the tool itself. The problem is that mocking and spies make it easy to not bother properly isolating the effects of a function for testing and then you end up having a test where 95% of it is setting up an elaborate array of mocks to create the condition you wish to test, which are completely incomprehensible to the next person trying to read it.
My second project at Google basically killed mocking for me and I've basically never done it since. Two things happened.
The first was that I worked on a rewrite of something (using GWT no less; it was more than a decade ago) and they decided to have a lot of test coverage and test requirements. That's fine but they way it was mandated and implemented, everybody just testing their service and DIed a bunch of mocks in.
The results were entirely predictable. The entire system was incredibly brittle and a service that existed for only 8 weeks behaved like legacy code. You could spend half a day fixing mocks in tests for a 30 minute change just because you switched backend services, changed the order of calls or just ended up calling a given service more times than expected. It was horrible and a complete waste of time.
Even the DI aspect of this was horrible because everything used Guice andd there wer emodules that installed modules that installed modules and modifying those to return mocks in a test environment was a massive effort that typically resulted in having a different environment (and injector) for test code vs production code so what are you actually testing?
The second was that about this time the Java engineers at the company went on a massive boondoggle to decide on whether to use (and mandate) EasyMock vs Mockito. This was additionally a waste of time. Regardless of the relative merits of either, there's really not that much difference. At no point is it worth completely changing your mocking framework in existing code. Who knows how many engineering man-yars were wasted on this.
Mocking encourages bad habits and a false sense of security. The solution is to have dummy versions of services and interfaces that have minimal correct behavior. So you might have a dummy Identity service that does simple lookups on an ID for permissions or metadata. If that's not what you're testing and you just need it to run a test, doing that with a mock is just wrong on so many levels.
I've basically never used mocks since, so much so that I find anyone who is strongly in favor of mocks or has strong opinions on mocking frameworks to be a huge red flag.
In part, you’re right, but there’s a practical difference between mocking and a good dummy version of a service. Take DynamoDB local as an example: you can insert items and they persist, delete items, delete tables, etc. Or in the Ruby on Rails world, one often would use SQLite as a local database for tests even if using a different DB in production.
Going further, there’s the whole test containers movement of having a real version of your dependency present for your tests. Of course, in a microservices world, bringing up the whole network of dependencies is extremely complicated and likely not warranted.
I'd go a bit farther — "mock" is basically the name for those dummy versions.
That said, there is a massive difference between writing mocks and using a mocking library like Mockito — just like there is a difference between using dependency injection and building your application around a DI framework.
> there is a massive difference between writing mocks and using a mocking library like Mockito
How to reconcile the differences in this discussion?
The comment at the root of the thread said "my experience with mocks is they were over-specified and lead to fragile services, even for fresh codebases. Using a 'fake' version of the service is better". The reply then said "if mocking doesn't provide a fake, it's not 'mocking'".
I'm wary of blanket sentiments like "if you ended up with a bad result, you weren't mocking". -- Is it the case that libraries like mockito are mostly used badly, but that correct use of them provides a good way of implementing robust 'fake services'?
The difference, IMO, between a mock and a proper "test" implementation is that traditionally a mock only exists to test interface boundaries, and the "implementation" is meant to be as much of a noop as possible. That's why the default behavior of almost any "automock" is to implement an interface by doing nothing and returning nothing (or perhaps default-initialized values) and provide tools for just tacking assertions onto it. If it was a proper implementation that just happened to be in-memory, it wouldn't really be a "mock", in my opinion.
For example, let's say you want to test that some handler is properly adding data to a cache. IMO the traditional mock approach that is supported by mocking libraries is to go take your RedisCache implementation and create a dummy that does nothing, then add assertions that say, the `set` method gets called with some set of arguments. You can add return values to the mock too, but I think this is mainly meant to be in service of just making the code run and not actually implementing anything.
Meanwhile, you could always make a minimal "test" implementation (I think these are sometimes called "fakes", traditionally, though I think this nomenclature is even more confusing) of your Cache interface that actually does behave like an in-memory cache, then your test could assert as to its contents. Doing this doesn't require a "mocking" library, and in this case, what you're making is not really a "mock" - it is, in fact, a full implementation of the interface, that you could use outside of tests (e.g. in a development server.) I think this can be a pretty good medium in some scenarios, especially since it plays along well with in-process tools like fake clocks/timers in languages like Go and JavaScript.
Despite the pitfalls, I mostly prefer to just use the actual implementations where possible, and for this I like testcontainers. Most webserver projects I write/work on naturally require a container runtime for development for other reasons, and testcontainers is glue that can use that existing container runtime setup (be it Docker or Podman) to pretty rapidly bootstrap test or dev service dependencies on-demand. With a little bit of manual effort, you can make it so that your normal test runner (e.g. `go test ./...`) can run tests normally, and automatically skip anything that requires a real service dependency in the event that there is no Docker socket available. (Though obviously, in a real setup, you'd also want a way to force the tests to be enabled, so that you can hopefully avoid an oopsie where CI isn't actually running your tests due to a regression.)
I think the argument they're making is that once you have this, you already have an easy way to test things that doesn't require bringing in an entire framework.
Mock is effective when developers keep applications at 4-5 layers deep (and honestly, you don't need more than that 97% of the time: initiators, controllers, services, transports, and cross cutting concerns).
The problem is, engineers love to solve problems, and the funnest types of problems are hypothetical ones!
Yep, I was guiltily of taking DI and writing a spider web of code. It's not the tools fault, it was my attitude. I _wanted_ something hard to work on, so I created something hard to work on. Nowadays, my team and I work on a 4-5 layer deep limit and our code base is tight, consistent, and has near 99% test coverage naturally. We use mock testing for testing the single class, and of course integration test for testing requirements. Not everyone will do it this way, and that's fine, but the most important thing is actually just creating a plan and sticking to it so as to be consistent.
In the end, don't blame the tool, when you (or your coworkers) simply lack discipline.
Absolutely the worst. 1000 line test setups that shatter into pieces the instant you try to make the simplest change to a function. Makes refactoring an absolute nightmare.
Which is absurd that people use mocks considering the tests are supposed to help with refactoring but because of the mocks they can’t make a change without breaking the test.
You've just built a calculator, and now you want to test it. Well, you have to push buttons on a calculator, so you build a robotic hand. And that hand needs a power source and some intelligence. And you need to photograph and OCR the result from the calculator screen.
This is kinda how we build software right? A little bit of "our logic" (calculation), represented as objects/actors/modules which "do things", but intermingled with million-LoC dependencies like databases and web servers.
After a while it gets frustrating setting up the robot hand and OCR equipment for each test case. Maybe it's just easier to test manually, or skip testing entirely.
At this point you can have an epiphany, and realise you only care about the numbers going in and out of the calculator, not the button pushes and pixels.
Mockito swoops in and prevents you from having that epiphany, by making it easier to keep doing things the stupid way.
Instead isolating the calculation from any IO, you can now write things like:
when(finger.pushbutton(1)).then(calculator.setState(1))
when(calculator.setAnswer(3)).then(camera.setOcr(3))
(I've mostly worked in Java, but it seems like other languages typically don't let you intercept calls and responses this way)
IMO Mockito is fine, the problem I’ve encountered is people taking the easy way out and trying to test something that needs a real integration test with a convoluted series of mocks.
I’ve been on projects where mocking _literally made the project less reliable_ because people ended up “testing” against mocks that didn’t accurately reflect the behavior of the real APIs.
It left us with functionality that wasn’t actually tested and resulted in real bugs and regressions that shipped.
Mocking is one of these weird programmer pop-culture memetic viruses that spread in the early 2000s and achieved complete victory in the 2010s, like Agile and OOP, and now there are entire generations of devs who it’s not that they’re making a bad or a poorly argued choice, it’s that they literally don’t even know there are other ways of thinking about these problems because these ideas have sucked all the oxygen out of the room.
I think there's room to argue "Agile" is a popular bastardisation of what's meant by "agile software development", and with "OOP" we got the lame Java interpretation rather than the sophisticated Smalltalk interpretation. -- Or I might think that these ideas aren't that good if their poor imitations win out over the "proper" ideas.
With mocking.. I'm willing to be curious that there's some good/effective way of doing it. But the idea of "you're just testing that the compiler works" comes to mind.
I’ll answer: Nothing specific to Mockito, it happens in every language. Tests “solidify” code which makes refactoring hard. And yet, after refactoring, one can be happy to have tests to check whether there is any regression.
Testing is hard. I’ve tried with AI today: No, it is still not capable of handling that kind of (straightforward) task (Using Claude).
They also encourage/enable code that is less testable. If you use mockito to get your fake responses/assertions where you need them, you don't have to think about your class's dependencies to make your code testable and therefore better decomposed. I don't even do TDD, but I still find that thinking about how I'd test a class guides me toward better-factored code.
One alternative to make code with typing styles in the Java way (as opposed to the Typescript or Go way) is to have a whole lot of custom interfaces and then you end up with a whole bunch of:
doTheThing(foo: Fooable) { ... }
when there's really only one Foo implementation in prod. It leads to (what feels like, to me) more code obfuscation in large projects, than the benefits that come out, at least for me.
So Mockito and friends are a nice alternative to that.
That is just my experience and opinion though, and there are definitely more valid or equally valid alternatives.
I don't think we have to choose. Naturally finding the "right division of labor" is as infinite as finding the "right level of abstraction", but I think the ideal situation is to strive toward code that is easy to test without having to introduce a lot of mocks or without infinite layers of abstraction.
Why? Every name you pick is likely to be weird in one language or another. Mockito does one thing well as a name, and that is hinting strongly at what it is (a mocking library).
>is likely to be weird in one language or another.
But this name is weird in the specific language it’s imitating (both the -ito termination for diminutives and the drink on which I assumed the name is based are Spanish).
This is a good opportunity to ditch mocking and use fakes with adapters. Not only mocks create brittle tests that often test only the framework itself, but they do so in an order of magnitude slower way.
Also, F Kotlin and their approach of "we'll reinvent the wheel with slightly different syntax and call it a new thing". Good riddance I say, let them implement their mockk, or whatever it is called, with ridiculous "fluent" syntax.
The whole mocks, fakes and the rest were always a terrible idea. Although people didn't realize it st the time, they were a band-aid to try to make poorly architected code testable.
90% of the time, needing to use a mock is one of the clearest code warning smells you have of there being an issue in the design your code.
It took a while, but the industry seems to be finally (although slowly) coming to this realization. And hopefully with it almost all of this can go away.
I agree. Tests relying on mocks rarely uncover or prevent issues. They also typically make it harder to make changes. Very bad idea that should have been left behind years ago.
90% of the time (or more): you don't. The real thing is perfectly fine in a test with the right setup. Fileio is fast, I just need a test file in a tempdir. databases are fast, I just need an easy way to settup my schema. Sometimes I need the isolation but normally I do not.
What you're describing is a very limited subset of testing, which presumably is fine for the projects you work on, but that experience does not generalise well.
Integration testing is of course useful, but generally one would want to create unit tests for every part of the code, and by definition it's not a unit test if hits multiple parts of the code simultaneously.
Apart from that, databases and file access may be fast but they still take resources and time to spin up; beyond a certain project and team size, it's far cheaper to mock those things. With a mock you can also easily simulate failure cases, bad data, etc. - how do you test for file access issues, or the database server being offline?
Using mocks properly is a sign of a well-factored codebase.
I worked on a project where a dev wanted to mock out the database in tests "for performance, because the database is slow". I almost lost my shit.
Even funnier, this was all hypothetical and yet taken as gospel. We hadn't even written the tests yet, so it was impossible to say whether they were slow or not. Nothing had been measured, no performance budget had been defined, no prototype of the supposedly slow tests had been written to demonstrate the point.
We ended up writing - no joke - less than 100 tests total, almost all of which hit the database, including some full integration tests, and the entire test suite finished in a few seconds.
I'm all for building in a way that respects performance as an engineering value, but we got lost somewhere along the way.
For the same reason you isolate variables in a scientific experiment; to ensure you're controlling the test that you're running, and not accidentally testing something else.
To easily simulate failure cases, a range of possible inputs, bad data etc.
To make the testing process faster when you have hundreds or thousands of tests, running on multiple builds simultaneously across an organisation.
| My personal take is that folks involved with the change severely underestimated the societal impact that it had. The fact that proper build support is non-existent to this day shows that agents are not a priority. That's okay if it isn't a priority, but when it was communicated with Mockito I perceived it as "Mockito is holding the JVM ecosystem back by using dynamic attachment, please switch immediately and figure it out on your own".
Id like to hear the platform team's perspective on this. As it stands, it is a pretty sad state of affairs that such a prominent library in the ecosystem was made out to be the scapegoat for the adoption of a platform change. It is not a healthy thing to treat the library maintainer community like this.
>Mockito 5 shipped a breaking change where its main artifact is now an agent. That's because starting JVM 22, the previous so-called "dynamic attachment of agents" is put behind a flag
Wouldn't this hold back enterprise adoption, the same way breaking changes meant that Java 8 was widely used for a long time?
IMO mockito has a relatively good use experience. If you use MockitoExtension, especially, you write code that is relatively maintainable, and easy to mutate. The problem without MockitoExtension is you can throw all kinds of junk in there and it just sits there doing nothing, without you knowing it.
Spy's on the other hand, are a pain in the neck. I think they should be good in theory, but in practice they are difficult, the debuggers really don't work correctly, setting breakpoints, stepping, etc, is just broken in many cases. Would love to know if this is just a difficult bug, or something that is baked into spying.
Most people want their test suite to pass. If they ugprade java and mockito prints out a message that they need to enabled '--some-flag' while running tests they're just going to add that flag to surefire in their pom. Seems like quite a small speedbump.
Tools for profiling, debugging, monitoring, thread analysis, and test coverage analysis can attach to the Java Virtual Machine (JVM) using the 'Tool Interface'
If you've got a running java process on your local machine right now, you can use 'jconsole' to see the stack traces of all threads, inspect various memory statistics, trigger an immediate garbage collection or heap dump, and so on. And of course, if the tool is an instrumenting profiler - it needs the power to modify the running code, to insert its instrumentation. Obviously you need certain permissions on the host to do this - just like attaching gdb to a running process.
This capability is used not just by for profiling, debugging and instrumentation but also by mockito to do its thing.
Java 21 introduced a warning [1] saying this will be disabled in a forthcoming version, unless the process is started with '-XX:+EnableDynamicAgentLoading' - whereas previously it was enabled by default and '-XX:+DisableAttachMechanism' was used to disable it.
The goal of doing this is "platform integrity" - preventing the attachment of debugging tools is useful in applications like DRM.
A JVM agent is able to instrument and modify running JVM applications. Stuff like debugging, hot patching, etc rely on this. You used to be able to tell the JVM to listen on a port where you could connect debuggers (agents) dynamically at runtime, but that was deemed a security issue so now you can only declare specific agents at launch time through command-line flags.
I respect the maintainer's decision, but I don't understand the justification.
> but when it was communicated with Mockito I perceived it as "Mockito is holding the JVM ecosystem back by using dynamic attachment, please switch immediately and figure it out on your own".
Who did the communication? Why is dynamic attachment through a flag a problem, and what was the solution? Why is "enable a flag when running tests" not a satisfactory solution? Why do you even need a _dynamic_ agent; don't you know ahead of time exactly what agent you need when using Mockito?
> While I fully understand the reasons that developers enjoy the feature richness of Kotlin as a programming language, its underlying implementation has significant downsides for projects like Mockito. Quite frankly, it's not fun to deal with.
Why support Kotlin in the first place? If it's a pain to deal with, perhaps the Kotlin user base is better served by a Kotlin-specific mocking framework, maintained by people who enjoy working on those Kotlin-specific code paths?
Mockito was indeed a poor fit for Kotlin. MockK is the one. Except I suppose for shops that have projects that mix Java and Kotlin and already have a Mockito tests.
Some complexities are discovered along the way, people don't know everything when they start.
They could also drop the support after some time, but then it would have created other set of problems for adoption and trustworthiness of the project.
Mockito is fine if you know how to write tests. You can write bad tests with Mockito or many other frameworks if that’s what you do. You can even write bad tests while programming in Rust from high up in your ivory tower. You can write good tests also.
> To me, it felt like the feature was presented as a done deal because of security.
Not security, but integrity, although security (which is the #1 concern of companies relying on a platform responsible for trillions of dollars) is certainly one of the primary motivations for integrity (others being performance, backward compatibility or "evolvability", and correctness). Integrity is the ability of code to locally declare its reliance on some invariant - e.g. that a certain class must not be extended, that a method can only be called by other methods in the same class, or that a field cannot be reassigned after being assigned in the constructor - and have the platform guarantee that the invariant is preserved globally throughout the lifetime of the program, no matter what other code does. What we call "memory safety" is an example of some invariants that have integrity.
This is obviously important for security as it significantly reduces the blast radius of a vulnerability (some attacks that can be done in JS or Python cannot be done in Java), but it's also important for performance, as the compiler needs to know that certain optimisations preserve meaning. E.g. strings cannot be constant-folded if they can't be relied upon to be truly immutable. It's also important for backward-compatibility or "evolvability", as libraries cannot depend on internals that are not explicitly exposed as public APIs; libraries doing that was the cause of the migration pain from Java 8 to 9+, as lots of libraries depended on internal, non-API methods that have changed when the JDK's evolution started picking up steam.
In Java, we've adopted a policy we call Integrity by Default (https://openjdk.org/jeps/8305968), which means that code in one component can violate invariants established by code in another component only if the application is made aware of it and allows it. What isn't allowed is for a library - which could be some fourth-level dependency - to decide for itself at some point during the program's execution, without the application's knowledge, that actually strings in this program should be mutable. We were, and are, open to any ideas as long as this principle is preserved.
Authors of components that do want to do such things find the policy inconvenient because their consumers need to do something extra that isn't required when using normal libraries. But this is a classic case of different users having conflicting requirements. No matter what you do, someone will be inconvenienced. We, the maintainers of the JDK, have opted for a solution that we believe minimises the pain and risk overall, when integrated over all users: Integrity is on by default, and components that wish to break it need an explicit configuration option to allow that.
> built on a solid foundation with ByteBuddy
ByteBuddy's author acknowledges that at least some aspects of ByteBuddy - and in particular the self-loading agent that Mockito used - weren't really a solid foundation, but now it should be: https://youtu.be/AzfhxgkBL9s?t=1843. We are grateful to Rafael for explaining his needs to us so that we could find a way to satisfy them without violating Integrity by Default.
Is there a point at which library maintainer feedback would meaningfully influence a by-default JVM change?
I keep a large production Java codebase and its deployments up-to-date. Short of upstreaming fixes to every major dependency, the only feasible way to continue upgrading JDK/JVM versions has often been to carry explicit exceptions to new defaults.
JPMS is a good example: --add-opens still remains valuable today for important infra like Hadoop, Spark, and Netty. If other, even more core projects (e.g. Arrow) hadn't modernized, the exceptions would be even more prolific.
If libraries so heavily depended upon like Mockito are unable to offer a viable alternative in response to JEP 451, my reaction would be to re-enable dynamic agent attachment rather than re-architect many years of test suites. I can't speak for others, but if this reaction holds broadly it would seem to defeat the point of by-default changes.
> Is there a point at which library maintainer feedback would meaningfully influence a by-default JVM change?
Of course, but keep in mind that all these changes were and are being done in response to feedback from other users, and we need to balance the requirements of mocking frameworks with those of people asking for better performance, better security, and better backward compatibility. When you have such a large ecosystem, users can have contradictory demands and sometimes it's impossible to satisfy everyone simultaneously. In those cases, we try to choose whatever we think will do the most good and the least harm over the entire ecosystem.
> JPMS is a good example: --add-opens still remains valuable today for important infra like Hadoop, Spark, and Netty. If other, even more core projects (e.g. Arrow) hadn't modernized, the exceptions would be even more prolific.
I think you have answered your own question. Make sure the libraries you rely on are well maintained, and if not - support them financially (actually, support them also if they are). BTW, I think that Netty is already in the process of abandoning its hacking of internals.
Anyone who has hacked internals agreed to a deal made in the notice we had in the internal files for many years prior to encapsulation [1], which was that the use of internals carries a commitment to added maintenance. Once they use the new supported mechanisms, that added burden is gone but they need to get there. I appreciate the fact that some open source projects are done by volunteers, and I think their users should compensate them, but they did enter into this deal voluntarily.
> If libraries so heavily depended upon like Mockito are unable to offer a viable alternative in response to JEP 451
The main "ergonomic" issue was lack of help from build tools like Gradle/Maven.
[1]: The notice was some version of:
WARNING: The contents of this source file are not part of any supported API.
Code that depends on them does so at its own risk: they are subject to change or removal without notice.
Of course, we did end up giving notice, usually a few years in advance, but no amount of time is sufficient for everyone. Note that JEP 451 is still in the warning period that started over two years ago (although probably not for long)
In September there was a supply-chain attack on NPM where the payload code injected hooks into the DOM API. Changing the behaviour of encapsulated components, like Java's standard library, is not possible now without the application explicitly allowing code to break the integrity of the encapsulated component.
I haven't read the discussion but this seems like the obvious answer considering the flag only needs to be set during test.
Presumably this might miss some edge case (where something else also needs the flag?) though an explicit allow of the mockito agent in the jvm arg would have solved for that.
Mocks make it easy to record and assert on method invocations. Additionally spys (instance mocks) are really useful when you need to forward to the real method or rely on some state.
At the moment I can't see anything Mokckito gives that you technically couldn't implement yourself via subclassing and overriding, but it'd be a lot of boilerplate to proxy things and record the arguments.
Subclasing and overriding is not a good idea. There is no compilation failure if you forget to override a function which can lead to flakey tests at best and prod data impact at worst.
Credentials should only be provided at the application root, which is going to be a different root for a test harness.
Mockito shouldn't change whether or not this is possible; the code shouldn't have the prod creds (or any external resource references) hard coded in the compiled bytecode.
I totally agree, I’m being tongue in cheek, but given how poor some codebases can be, the more precautions the better ie compilation failures on non-mocked functions.
Mockito allows one to write mocks in tests for code that doesn't use dependency injection and isn't properly testable in any other way.
On the one hand, you should just design things to be testable from the start. On the other... I'm already working in this codebase with 20 years of legacy untestable design...
Google API libraries mark every class as "final" so it's not trivial to mock-extend it for tests. But third-party IO is exactly the thing you'd want to mock.
Probably because they zealously followed "Effective Java" book.
Once you start writing adapters you need a way to instantiate them to choose between implementations, and factories is often used for this. Then you might generalize the test suites to make the setup easier and you end up with the infamous FactoryFactory pattern.
No, some other library classes accept only their own, not my adapter.
Not mentioning of course needless copy-pasting dosens of members in the adapter. And it must be in prod code, not tests, even though it's documentation would say "Adapter for X, exists only for tests, to be able to mock X".
Mockito uses declarative matching style of specifying what should be mocked. You don't need to implement or even stub all of interface methods since Mockito can do it itself. It may be extremely concise. For example, interfaces may have tens methods or even more, but only one method is needed (say, java.sql.ResultSet). And finally probably the most important thing, interaction with mocks is recorded and then can be verified if certain methods were invoked with certain arguments.
That’s the seductive power of mocking - you get a test up and running quickly. The benefit to the initial test writer is significant.
The cost is the pain - sometimes nightmarish - for other contributors to the code base since tests depending on mocking are far more brittle.
Someone changes code to check if the ResultSet is empty before further processing and a large number of your mock based tests break as the original test author will only have mocked enough of the class to support the current implementation.
Working on a 10+ year old code base, making a small simple safe change and then seeing a bunch of unit tests fail, my reaction is always “please let the failing tests not rely on mocks”.
The point is to let you create mocks without having to go through the whole polymorphism rigmarole, without forcing classes to define a separate interface or anything like that.
because even supposing you have an interface for your thing under test (which you don't necessarily, nor do you necessarily want to have to) it lets you skip over having to do any fake implementations, have loads of variations of said fake implementations, have that code live somewhere, etc etc.
Instead your mocks are all just inline in the test code: ephemeral, basically declarative therefore readily readable & grokable without too much diversion, and easily changed.
A really good usecase for Java's 'Reflection' feature.
An anonymous inner class is also ephemeral, declarative, inline, capable of extending as well as implementing, and readily readable. What it isn't is terse.
Mocking's killer feature is the ability to partially implement/extend by having some default that makes some sense in a testing situation and is easily instantiable without calling a super constructor.
Magicmock in python is the single best mocking library though, too many times have I really wanted mockito to also default to returning a mock instead of null.
Yeah, it's funny, I'm often arguing in the corner of being verbose in the name of plain-ness and greater simplicity.
I realise it's subjective, but this is one of the rare cases where I think the opposite is true, and using the 'magic' thing that shortcuts language primitives in a sort-of DSL is actually the better choice.
It's dumb, it's one or two lines, it says what it does, there's almost zero diversion. Sure you can do it by other means but I think the (what I will claim is) 'truly' inline style code of Mockito is actually a material value add in readability & grokability if you're just trying to debug a failing test you haven't seen in ages, which is basically the usecase I have in mind whenever writing test code.
As someone who has been out of Java for close to 10 years now, you certainly could do without Mockito, but you'd be writing a lot of boiler plate code repetitively. There's also the case of third-party libraries that you don't control and Mockito has decent facilities for working with those, especially when you're working with a codebase that isn't pure DI and interfaces.
There are many cases where you don't control the library code your code depends on that you want to test. Also, the FactoryFactoryFactory patterns can be quite cumbersome and simply mocking out something makes for a far simpler test. There are likely more common cases.
Before Mockito, it was common (where I worked) to create an interface just to support testing. This is an anti-pattern in my opinion. To create interfaces just for testing complicates the code and it is one of my pet peeves. It also encourages the factory pattern.
Sad to see an important project's core maintainer leave but their justification seems very understandable. It is sad so much of OSS is maintained by very few as they alluded to in the XKCD comment, an especially given they felt the JVM ecosystem was causing them pain with limited support or feedback possible. I think it is always a little irresponsible to cause a great deal of breakage and not be there to support those who you break downstream of your project.
So funny, he essentially works for free for 10 years, then finally burns out because he doesn't want to put up with a bunch of annoying work? This is why you shouldn't work on open source unless you have a business strategy to get paid. Tons of stuff in life is 100x more annoying and exhausting if you aren't making any money. If he was making $1 million per year from this I doubt his energy would be drained.
I wouldn't make such a conclusion. I don't think there is any info about whether OP got financial incentives for his work or not. In fact, he posted on Mastodon, he's gonna be doing open source Rust work further on.
I've been using Mockito for about 4 years, all in Kotlin. I always found it to be "plenty good" for like 99% of the cases I needed it; and things more complicated or confusing or messy were usually my fault (poor separation of concerns, etc).
I regularly found it quite helpful in both its spy() and mock() functionality.
I never found it meaningfully more or less useful than MockK, though I have heard MockK is the "one that's better for Kotlin". It's mostly just vocabulary changes for me, the user.
I'm going to have to monitor Mockito's future and see if I'll need to swap to MockK at some point if Mockito becomes unmaintained.
If I was OP I'd retire happy knowing that a very thankless job is well done! Given what it does: the more outrage the better. Projects like Mockito call out the lazy and indolent for whom they are and the hissing and spitting in return can simply be laughed at.
10 years this bloke has given his time and effort to help people. He states: nearly a third of his life.
I'll raise a glass and say "was hale" or perhaps wassail as an Englander might.
1. Kotlin is a hack and 2. Rust is more fun.
Pretty understandable why one would simply want to move on to greener pastures.
Here are some examples that have hit the graveyard: It's been 2 years since exception handling in switch was proposed [0], 3 years since null-restricted types were proposed [1], 4 years since string templates [2], 8 years since concise method bodies [3], and 11 years for JSON parsing [4].
[0] https://inside.java/2023/12/15/switch-case-effect/
[1] https://openjdk.org/jeps/8303099
[2] https://openjdk.org/jeps/430
[3] https://openjdk.org/jeps/8209434
[4] https://openjdk.org/jeps/198
The first was that I worked on a rewrite of something (using GWT no less; it was more than a decade ago) and they decided to have a lot of test coverage and test requirements. That's fine but they way it was mandated and implemented, everybody just testing their service and DIed a bunch of mocks in.
The results were entirely predictable. The entire system was incredibly brittle and a service that existed for only 8 weeks behaved like legacy code. You could spend half a day fixing mocks in tests for a 30 minute change just because you switched backend services, changed the order of calls or just ended up calling a given service more times than expected. It was horrible and a complete waste of time.
Even the DI aspect of this was horrible because everything used Guice andd there wer emodules that installed modules that installed modules and modifying those to return mocks in a test environment was a massive effort that typically resulted in having a different environment (and injector) for test code vs production code so what are you actually testing?
The second was that about this time the Java engineers at the company went on a massive boondoggle to decide on whether to use (and mandate) EasyMock vs Mockito. This was additionally a waste of time. Regardless of the relative merits of either, there's really not that much difference. At no point is it worth completely changing your mocking framework in existing code. Who knows how many engineering man-yars were wasted on this.
Mocking encourages bad habits and a false sense of security. The solution is to have dummy versions of services and interfaces that have minimal correct behavior. So you might have a dummy Identity service that does simple lookups on an ID for permissions or metadata. If that's not what you're testing and you just need it to run a test, doing that with a mock is just wrong on so many levels.
I've basically never used mocks since, so much so that I find anyone who is strongly in favor of mocks or has strong opinions on mocking frameworks to be a huge red flag.
If you aren’t doing this with mocks then you’re doing mocks wrong.
Going further, there’s the whole test containers movement of having a real version of your dependency present for your tests. Of course, in a microservices world, bringing up the whole network of dependencies is extremely complicated and likely not warranted.
That said, there is a massive difference between writing mocks and using a mocking library like Mockito — just like there is a difference between using dependency injection and building your application around a DI framework.
How to reconcile the differences in this discussion?
The comment at the root of the thread said "my experience with mocks is they were over-specified and lead to fragile services, even for fresh codebases. Using a 'fake' version of the service is better". The reply then said "if mocking doesn't provide a fake, it's not 'mocking'".
I'm wary of blanket sentiments like "if you ended up with a bad result, you weren't mocking". -- Is it the case that libraries like mockito are mostly used badly, but that correct use of them provides a good way of implementing robust 'fake services'?
For example, let's say you want to test that some handler is properly adding data to a cache. IMO the traditional mock approach that is supported by mocking libraries is to go take your RedisCache implementation and create a dummy that does nothing, then add assertions that say, the `set` method gets called with some set of arguments. You can add return values to the mock too, but I think this is mainly meant to be in service of just making the code run and not actually implementing anything.
Meanwhile, you could always make a minimal "test" implementation (I think these are sometimes called "fakes", traditionally, though I think this nomenclature is even more confusing) of your Cache interface that actually does behave like an in-memory cache, then your test could assert as to its contents. Doing this doesn't require a "mocking" library, and in this case, what you're making is not really a "mock" - it is, in fact, a full implementation of the interface, that you could use outside of tests (e.g. in a development server.) I think this can be a pretty good medium in some scenarios, especially since it plays along well with in-process tools like fake clocks/timers in languages like Go and JavaScript.
Despite the pitfalls, I mostly prefer to just use the actual implementations where possible, and for this I like testcontainers. Most webserver projects I write/work on naturally require a container runtime for development for other reasons, and testcontainers is glue that can use that existing container runtime setup (be it Docker or Podman) to pretty rapidly bootstrap test or dev service dependencies on-demand. With a little bit of manual effort, you can make it so that your normal test runner (e.g. `go test ./...`) can run tests normally, and automatically skip anything that requires a real service dependency in the event that there is no Docker socket available. (Though obviously, in a real setup, you'd also want a way to force the tests to be enabled, so that you can hopefully avoid an oopsie where CI isn't actually running your tests due to a regression.)
The problem is, engineers love to solve problems, and the funnest types of problems are hypothetical ones!
Yep, I was guiltily of taking DI and writing a spider web of code. It's not the tools fault, it was my attitude. I _wanted_ something hard to work on, so I created something hard to work on. Nowadays, my team and I work on a 4-5 layer deep limit and our code base is tight, consistent, and has near 99% test coverage naturally. We use mock testing for testing the single class, and of course integration test for testing requirements. Not everyone will do it this way, and that's fine, but the most important thing is actually just creating a plan and sticking to it so as to be consistent.
In the end, don't blame the tool, when you (or your coworkers) simply lack discipline.
This is kinda how we build software right? A little bit of "our logic" (calculation), represented as objects/actors/modules which "do things", but intermingled with million-LoC dependencies like databases and web servers.
After a while it gets frustrating setting up the robot hand and OCR equipment for each test case. Maybe it's just easier to test manually, or skip testing entirely.
At this point you can have an epiphany, and realise you only care about the numbers going in and out of the calculator, not the button pushes and pixels.
Mockito swoops in and prevents you from having that epiphany, by making it easier to keep doing things the stupid way.
Instead isolating the calculation from any IO, you can now write things like: when(finger.pushbutton(1)).then(calculator.setState(1)) when(calculator.setAnswer(3)).then(camera.setOcr(3))
(I've mostly worked in Java, but it seems like other languages typically don't let you intercept calls and responses this way)
I’ve been on projects where mocking _literally made the project less reliable_ because people ended up “testing” against mocks that didn’t accurately reflect the behavior of the real APIs.
It left us with functionality that wasn’t actually tested and resulted in real bugs and regressions that shipped.
Mocking is one of these weird programmer pop-culture memetic viruses that spread in the early 2000s and achieved complete victory in the 2010s, like Agile and OOP, and now there are entire generations of devs who it’s not that they’re making a bad or a poorly argued choice, it’s that they literally don’t even know there are other ways of thinking about these problems because these ideas have sucked all the oxygen out of the room.
Ha.
I think there's room to argue "Agile" is a popular bastardisation of what's meant by "agile software development", and with "OOP" we got the lame Java interpretation rather than the sophisticated Smalltalk interpretation. -- Or I might think that these ideas aren't that good if their poor imitations win out over the "proper" ideas.
With mocking.. I'm willing to be curious that there's some good/effective way of doing it. But the idea of "you're just testing that the compiler works" comes to mind.
Testing is hard. I’ve tried with AI today: No, it is still not capable of handling that kind of (straightforward) task (Using Claude).
So Mockito and friends are a nice alternative to that.
That is just my experience and opinion though, and there are definitely more valid or equally valid alternatives.
But this name is weird in the specific language it’s imitating (both the -ito termination for diminutives and the drink on which I assumed the name is based are Spanish).
Also, F Kotlin and their approach of "we'll reinvent the wheel with slightly different syntax and call it a new thing". Good riddance I say, let them implement their mockk, or whatever it is called, with ridiculous "fluent" syntax.
90% of the time, needing to use a mock is one of the clearest code warning smells you have of there being an issue in the design your code.
It took a while, but the industry seems to be finally (although slowly) coming to this realization. And hopefully with it almost all of this can go away.
What you're describing is a very limited subset of testing, which presumably is fine for the projects you work on, but that experience does not generalise well.
Integration testing is of course useful, but generally one would want to create unit tests for every part of the code, and by definition it's not a unit test if hits multiple parts of the code simultaneously.
Apart from that, databases and file access may be fast but they still take resources and time to spin up; beyond a certain project and team size, it's far cheaper to mock those things. With a mock you can also easily simulate failure cases, bad data, etc. - how do you test for file access issues, or the database server being offline?
Using mocks properly is a sign of a well-factored codebase.
Even funnier, this was all hypothetical and yet taken as gospel. We hadn't even written the tests yet, so it was impossible to say whether they were slow or not. Nothing had been measured, no performance budget had been defined, no prototype of the supposedly slow tests had been written to demonstrate the point.
We ended up writing - no joke - less than 100 tests total, almost all of which hit the database, including some full integration tests, and the entire test suite finished in a few seconds.
I'm all for building in a way that respects performance as an engineering value, but we got lost somewhere along the way.
Hopefully one day you'll back at that, and realise what an immature attitude that was.
To easily simulate failure cases, a range of possible inputs, bad data etc.
To make the testing process faster when you have hundreds or thousands of tests, running on multiple builds simultaneously across an organisation.
Off the top of my head :-)
I don't know why owners/maintainers show grace like this.
For me; I would say I'm done and hand it over or just archive the repo.
Hope Tim finds a measure of sanity after he steps down.
Id like to hear the platform team's perspective on this. As it stands, it is a pretty sad state of affairs that such a prominent library in the ecosystem was made out to be the scapegoat for the adoption of a platform change. It is not a healthy thing to treat the library maintainer community like this.
Wouldn't this hold back enterprise adoption, the same way breaking changes meant that Java 8 was widely used for a long time?
Most places are just about getting rid of 8 for 17.
Spy's on the other hand, are a pain in the neck. I think they should be good in theory, but in practice they are difficult, the debuggers really don't work correctly, setting breakpoints, stepping, etc, is just broken in many cases. Would love to know if this is just a difficult bug, or something that is baked into spying.
Ok am I being stupid or is the pragmatic solution not to just to enable this flag for test runs etc. and leave it off in prod?
If you've got a running java process on your local machine right now, you can use 'jconsole' to see the stack traces of all threads, inspect various memory statistics, trigger an immediate garbage collection or heap dump, and so on. And of course, if the tool is an instrumenting profiler - it needs the power to modify the running code, to insert its instrumentation. Obviously you need certain permissions on the host to do this - just like attaching gdb to a running process.
This capability is used not just by for profiling, debugging and instrumentation but also by mockito to do its thing.
Java 21 introduced a warning [1] saying this will be disabled in a forthcoming version, unless the process is started with '-XX:+EnableDynamicAgentLoading' - whereas previously it was enabled by default and '-XX:+DisableAttachMechanism' was used to disable it.
The goal of doing this is "platform integrity" - preventing the attachment of debugging tools is useful in applications like DRM.
[1] https://openjdk.org/jeps/451
> but when it was communicated with Mockito I perceived it as "Mockito is holding the JVM ecosystem back by using dynamic attachment, please switch immediately and figure it out on your own".
Who did the communication? Why is dynamic attachment through a flag a problem, and what was the solution? Why is "enable a flag when running tests" not a satisfactory solution? Why do you even need a _dynamic_ agent; don't you know ahead of time exactly what agent you need when using Mockito?
> While I fully understand the reasons that developers enjoy the feature richness of Kotlin as a programming language, its underlying implementation has significant downsides for projects like Mockito. Quite frankly, it's not fun to deal with.
Why support Kotlin in the first place? If it's a pain to deal with, perhaps the Kotlin user base is better served by a Kotlin-specific mocking framework, maintained by people who enjoy working on those Kotlin-specific code paths?
Some complexities are discovered along the way, people don't know everything when they start.
They could also drop the support after some time, but then it would have created other set of problems for adoption and trustworthiness of the project.
Not security, but integrity, although security (which is the #1 concern of companies relying on a platform responsible for trillions of dollars) is certainly one of the primary motivations for integrity (others being performance, backward compatibility or "evolvability", and correctness). Integrity is the ability of code to locally declare its reliance on some invariant - e.g. that a certain class must not be extended, that a method can only be called by other methods in the same class, or that a field cannot be reassigned after being assigned in the constructor - and have the platform guarantee that the invariant is preserved globally throughout the lifetime of the program, no matter what other code does. What we call "memory safety" is an example of some invariants that have integrity.
This is obviously important for security as it significantly reduces the blast radius of a vulnerability (some attacks that can be done in JS or Python cannot be done in Java), but it's also important for performance, as the compiler needs to know that certain optimisations preserve meaning. E.g. strings cannot be constant-folded if they can't be relied upon to be truly immutable. It's also important for backward-compatibility or "evolvability", as libraries cannot depend on internals that are not explicitly exposed as public APIs; libraries doing that was the cause of the migration pain from Java 8 to 9+, as lots of libraries depended on internal, non-API methods that have changed when the JDK's evolution started picking up steam.
In Java, we've adopted a policy we call Integrity by Default (https://openjdk.org/jeps/8305968), which means that code in one component can violate invariants established by code in another component only if the application is made aware of it and allows it. What isn't allowed is for a library - which could be some fourth-level dependency - to decide for itself at some point during the program's execution, without the application's knowledge, that actually strings in this program should be mutable. We were, and are, open to any ideas as long as this principle is preserved.
Authors of components that do want to do such things find the policy inconvenient because their consumers need to do something extra that isn't required when using normal libraries. But this is a classic case of different users having conflicting requirements. No matter what you do, someone will be inconvenienced. We, the maintainers of the JDK, have opted for a solution that we believe minimises the pain and risk overall, when integrated over all users: Integrity is on by default, and components that wish to break it need an explicit configuration option to allow that.
> built on a solid foundation with ByteBuddy
ByteBuddy's author acknowledges that at least some aspects of ByteBuddy - and in particular the self-loading agent that Mockito used - weren't really a solid foundation, but now it should be: https://youtu.be/AzfhxgkBL9s?t=1843. We are grateful to Rafael for explaining his needs to us so that we could find a way to satisfy them without violating Integrity by Default.
I keep a large production Java codebase and its deployments up-to-date. Short of upstreaming fixes to every major dependency, the only feasible way to continue upgrading JDK/JVM versions has often been to carry explicit exceptions to new defaults.
JPMS is a good example: --add-opens still remains valuable today for important infra like Hadoop, Spark, and Netty. If other, even more core projects (e.g. Arrow) hadn't modernized, the exceptions would be even more prolific.
If libraries so heavily depended upon like Mockito are unable to offer a viable alternative in response to JEP 451, my reaction would be to re-enable dynamic agent attachment rather than re-architect many years of test suites. I can't speak for others, but if this reaction holds broadly it would seem to defeat the point of by-default changes.
Of course, but keep in mind that all these changes were and are being done in response to feedback from other users, and we need to balance the requirements of mocking frameworks with those of people asking for better performance, better security, and better backward compatibility. When you have such a large ecosystem, users can have contradictory demands and sometimes it's impossible to satisfy everyone simultaneously. In those cases, we try to choose whatever we think will do the most good and the least harm over the entire ecosystem.
> JPMS is a good example: --add-opens still remains valuable today for important infra like Hadoop, Spark, and Netty. If other, even more core projects (e.g. Arrow) hadn't modernized, the exceptions would be even more prolific.
I think you have answered your own question. Make sure the libraries you rely on are well maintained, and if not - support them financially (actually, support them also if they are). BTW, I think that Netty is already in the process of abandoning its hacking of internals.
Anyone who has hacked internals agreed to a deal made in the notice we had in the internal files for many years prior to encapsulation [1], which was that the use of internals carries a commitment to added maintenance. Once they use the new supported mechanisms, that added burden is gone but they need to get there. I appreciate the fact that some open source projects are done by volunteers, and I think their users should compensate them, but they did enter into this deal voluntarily.
> If libraries so heavily depended upon like Mockito are unable to offer a viable alternative in response to JEP 451
But they have, and we advised them on how: https://github.com/mockito/mockito/issues/3037
The main "ergonomic" issue was lack of help from build tools like Gradle/Maven.
[1]: The notice was some version of:
Of course, we did end up giving notice, usually a few years in advance, but no amount of time is sufficient for everyone. Note that JEP 451 is still in the warning period that started over two years ago (although probably not for long)Examples?
Presumably this might miss some edge case (where something else also needs the flag?) though an explicit allow of the mockito agent in the jvm arg would have solved for that.
You can and should explicitly specify Mockito as an agent in the JVM configuration, as it is one.
At the moment I can't see anything Mokckito gives that you technically couldn't implement yourself via subclassing and overriding, but it'd be a lot of boilerplate to proxy things and record the arguments.
Mockito shouldn't change whether or not this is possible; the code shouldn't have the prod creds (or any external resource references) hard coded in the compiled bytecode.
On the one hand, you should just design things to be testable from the start. On the other... I'm already working in this codebase with 20 years of legacy untestable design...
Probably because they zealously followed "Effective Java" book.
You write an adapter.
Not mentioning of course needless copy-pasting dosens of members in the adapter. And it must be in prod code, not tests, even though it's documentation would say "Adapter for X, exists only for tests, to be able to mock X".
The cost is the pain - sometimes nightmarish - for other contributors to the code base since tests depending on mocking are far more brittle.
Someone changes code to check if the ResultSet is empty before further processing and a large number of your mock based tests break as the original test author will only have mocked enough of the class to support the current implementation.
Working on a 10+ year old code base, making a small simple safe change and then seeing a bunch of unit tests fail, my reaction is always “please let the failing tests not rely on mocks”.
Instead your mocks are all just inline in the test code: ephemeral, basically declarative therefore readily readable & grokable without too much diversion, and easily changed.
A really good usecase for Java's 'Reflection' feature.
Mocking's killer feature is the ability to partially implement/extend by having some default that makes some sense in a testing situation and is easily instantiable without calling a super constructor.
Magicmock in python is the single best mocking library though, too many times have I really wanted mockito to also default to returning a mock instead of null.
Yeah, it's funny, I'm often arguing in the corner of being verbose in the name of plain-ness and greater simplicity.
I realise it's subjective, but this is one of the rare cases where I think the opposite is true, and using the 'magic' thing that shortcuts language primitives in a sort-of DSL is actually the better choice.
It's dumb, it's one or two lines, it says what it does, there's almost zero diversion. Sure you can do it by other means but I think the (what I will claim is) 'truly' inline style code of Mockito is actually a material value add in readability & grokability if you're just trying to debug a failing test you haven't seen in ages, which is basically the usecase I have in mind whenever writing test code.
I prefer Mockito's approach.
So funny, he essentially works for free for 10 years, then finally burns out because he doesn't want to put up with a bunch of annoying work? This is why you shouldn't work on open source unless you have a business strategy to get paid. Tons of stuff in life is 100x more annoying and exhausting if you aren't making any money. If he was making $1 million per year from this I doubt his energy would be drained.
People quit all the time from highly paid jobs due to burnout. This absolutely does not follow.