Details

    • Type: Bug Bug
    • Status: Resolved Resolved
    • Priority: Major Major
    • Resolution: Works as Designed
    • Affects Version/s: 3.0.0.M3
    • Fix Version/s: None
    • Component/s: GRAILS
    • Labels:
      None
    • Environment:

      Mac OS X 10.7.4

      Description

      Grails Integration tests that extend grails.plugin.spock.IntegrationSpec do not run properly when using the "Run As->JUnit Test". If I change the extends to be spock.lang.Specification, then it runs but that class doesn't provide the proper test support.

        Activity

        Hide
        Kris De Volder (c) added a comment -

        I think you probably have to run those tests via the grails 'test-app' command only.

        When you "Run As Junit test" you are just using a plain Eclipse Java/Junit test runner. This runner knows nothing of Grails infrastructure. You can get away with this only for real 'unit tests' that don't require special infrastructure/setup from Grails.

        So to run your tests try "Run As >> Grails command (test-app)".

        Kris

        Show
        Kris De Volder (c) added a comment - I think you probably have to run those tests via the grails 'test-app' command only. When you "Run As Junit test" you are just using a plain Eclipse Java/Junit test runner. This runner knows nothing of Grails infrastructure. You can get away with this only for real 'unit tests' that don't require special infrastructure/setup from Grails. So to run your tests try "Run As >> Grails command (test-app)". Kris
        Hide
        Jean Duteau added a comment -

        But my unit tests are doing special infrastructure/setup from Grails. And if I change to extending the Specification base class, quite a bit of that setup occurs. As well, I have created a non-Spock unit test that has exactly the same code and it runs just fine. So Spock tests run as unit tests, JUnit tests run as integration tests, but Spock tests do not run as integration tests. And my hypothesis is that it has something to do with the IntegrationSpec class.

        Show
        Jean Duteau added a comment - But my unit tests are doing special infrastructure/setup from Grails. And if I change to extending the Specification base class, quite a bit of that setup occurs. As well, I have created a non-Spock unit test that has exactly the same code and it runs just fine. So Spock tests run as unit tests, JUnit tests run as integration tests, but Spock tests do not run as integration tests. And my hypothesis is that it has something to do with the IntegrationSpec class.
        Hide
        Kris De Volder (c) added a comment -

        Hi Jean,

        I don't truly understand the Grails test running infrastructure details. But because you say your tests are 'integration' tests I would not expect them to run with "Run as Junit".

        Here is what I know: since Grails 2.0 the test insfrastucture has more advanced mocking capability. That lets you run many more tests as plain 'unit' tests instead of 'integration' tests. These unit tests don't really use the grails runtime infrastucture as it would be available during a run-app. Instead they use some mocked versions of the infrastructure.

        Grails distinguishes between 'unit' and 'integration' test. You should only expect to be able to run 'unit' test with the Eclipse "Run as Junit" runner.
        If your test is a unit test, and it doesn't "Run As Junit" I'd be inclined to indeed call that a bug. We'd then have to determine if its an STS bug or a Grails bug.

        But if your test is an "integration" test then I'd be inclined to close this bug report as "Works as Designed".

        So the first question is whether you can run your tests successfully with the grails test-app on the commandline, specifically as a "unit" test.

        See here http://grails.org/doc/latest/ref/Command%20Line/test-app.html
        For some info on test-app command. Note the options to restrict to running "unit:" version "integration:" test. So please try running your test with grails test-app as a "unit" test and see if that actually works or not. That will give us useful info to determine the next course of action.

        Show
        Kris De Volder (c) added a comment - Hi Jean, I don't truly understand the Grails test running infrastructure details. But because you say your tests are 'integration' tests I would not expect them to run with "Run as Junit". Here is what I know: since Grails 2.0 the test insfrastucture has more advanced mocking capability. That lets you run many more tests as plain 'unit' tests instead of 'integration' tests. These unit tests don't really use the grails runtime infrastucture as it would be available during a run-app. Instead they use some mocked versions of the infrastructure. Grails distinguishes between 'unit' and 'integration' test. You should only expect to be able to run 'unit' test with the Eclipse "Run as Junit" runner. If your test is a unit test, and it doesn't "Run As Junit" I'd be inclined to indeed call that a bug. We'd then have to determine if its an STS bug or a Grails bug. But if your test is an "integration" test then I'd be inclined to close this bug report as "Works as Designed". So the first question is whether you can run your tests successfully with the grails test-app on the commandline, specifically as a "unit" test. See here http://grails.org/doc/latest/ref/Command%20Line/test-app.html For some info on test-app command. Note the options to restrict to running "unit:" version "integration:" test. So please try running your test with grails test-app as a "unit" test and see if that actually works or not. That will give us useful info to determine the next course of action.
        Hide
        Jean Duteau added a comment -

        A Grails integration test actually hits the database as opposed to a unit test where everything but the class in question is mocked out. And STS does run integration tests as long as they are JUnit-style tests. I really think that the unit/integration distinction is a red herring here. I have a problem with running Spock-style integration tests. Here is what I know with STS:

        a) I can run JUnit-style unit tests within STS
        b) I can run Spock-style unit tests within STS
        c) I can run JUnit-style integration tests within STS
        d) I can not run Spock-style integration tests within STS

        I took the Spock integration test that I have and made it a JUnit-style integration test and it runs just fine in STS. So STS can handle Grails integration tests.

        For now, as a workaround, I'm writing my unit tests as Spock tests and I'm writing my integration tests as JUnit tests.

        I do run all of my tests on the command-line and they all work - even my Spock integration test. I can't run my integration tests as unit tests because they are testing actually hitting the database and other infrastructure stuff.

        Show
        Jean Duteau added a comment - A Grails integration test actually hits the database as opposed to a unit test where everything but the class in question is mocked out. And STS does run integration tests as long as they are JUnit-style tests. I really think that the unit/integration distinction is a red herring here. I have a problem with running Spock-style integration tests. Here is what I know with STS: a) I can run JUnit-style unit tests within STS b) I can run Spock-style unit tests within STS c) I can run JUnit-style integration tests within STS d) I can not run Spock-style integration tests within STS I took the Spock integration test that I have and made it a JUnit-style integration test and it runs just fine in STS. So STS can handle Grails integration tests. For now, as a workaround, I'm writing my unit tests as Spock tests and I'm writing my integration tests as JUnit tests. I do run all of my tests on the command-line and they all work - even my Spock integration test. I can't run my integration tests as unit tests because they are testing actually hitting the database and other infrastructure stuff.
        Hide
        Kris De Volder (c) added a comment - - edited

        > I took the Spock integration test that I have and made it a JUnit-style integration test and it runs
        > just fine in STS.

        OK, that is interesting.

        > So STS can handle Grails integration tests.

        Well... I guess it depends. Like I said I don't truly understand what all the things 'test-app' sets up for you in your execution context as opposed to what part of it is setup explicitly by code in your test classes / superclasses etc.

        But the fact is that when you use 'Run as Junit'. You are not getting any of the setup grails does implicitly because what you are using is just a 'naked' Junit test runner. Nothing in it is grails specific. Any 'infrastructure' that is still being setup must therefore somehow be present in your test classes themselves, maybe inherited from a super class, or via an AST transform or etc.

        So I guess, you are right. Maybe whether a test is classed as integration or unit is less important than whether or not it actually depends on some implicit context that grails will setup only for integration tests. You may get lucky, if you don't use the things that aren't there in the 'naked' JUnit runner, it may work.

        Still it would be interesting to better understand/determine precisely why your case (d) is not working. I'm willing to go 'down this rathole' with you. But I have to warn you ahead of time that I expect it to be a bit of work playing detective... and at the end there's no guarantee that I can fix it. (but without knowing what is the exact cause we are just making wild speculations at this point).

        Do you think you could create a small project with a sample test that shows the problem? Then I can play with that and hopefully gain some understanding of why it breaks?

        Kris

        Show
        Kris De Volder (c) added a comment - - edited > I took the Spock integration test that I have and made it a JUnit-style integration test and it runs > just fine in STS. OK, that is interesting. > So STS can handle Grails integration tests. Well... I guess it depends. Like I said I don't truly understand what all the things 'test-app' sets up for you in your execution context as opposed to what part of it is setup explicitly by code in your test classes / superclasses etc. But the fact is that when you use 'Run as Junit'. You are not getting any of the setup grails does implicitly because what you are using is just a 'naked' Junit test runner. Nothing in it is grails specific. Any 'infrastructure' that is still being setup must therefore somehow be present in your test classes themselves, maybe inherited from a super class, or via an AST transform or etc. So I guess, you are right. Maybe whether a test is classed as integration or unit is less important than whether or not it actually depends on some implicit context that grails will setup only for integration tests. You may get lucky, if you don't use the things that aren't there in the 'naked' JUnit runner, it may work. Still it would be interesting to better understand/determine precisely why your case (d) is not working. I'm willing to go 'down this rathole' with you. But I have to warn you ahead of time that I expect it to be a bit of work playing detective... and at the end there's no guarantee that I can fix it. (but without knowing what is the exact cause we are just making wild speculations at this point). Do you think you could create a small project with a sample test that shows the problem? Then I can play with that and hopefully gain some understanding of why it breaks? Kris
        Hide
        Kris De Volder (c) added a comment -

        > I took the Spock integration test that I have and made it a JUnit-style integration test and it runs
        > just fine in STS. So STS can handle Grails integration tests.

        I'd be especially interested in seeing those two side-by-side. If you can manage a test-project like that.

        Show
        Kris De Volder (c) added a comment - > I took the Spock integration test that I have and made it a JUnit-style integration test and it runs > just fine in STS. So STS can handle Grails integration tests. I'd be especially interested in seeing those two side-by-side. If you can manage a test-project like that.
        Hide
        Jean Duteau added a comment - - edited

        Just finished an all-night debugging session on my real app. I've now got a breather and will create a small sample project that exhibits the problem behaviour.

        Show
        Jean Duteau added a comment - - edited Just finished an all-night debugging session on my real app. I've now got a breather and will create a small sample project that exhibits the problem behaviour.
        Hide
        Jean Duteau added a comment -

        This one can be closed because I think I was mocking some things out in the tests. Your initial belief was correct. When I created an empty new project, I got the behaviour that you suggested - integration tests only run with test-app and not within STS.

        Show
        Jean Duteau added a comment - This one can be closed because I think I was mocking some things out in the tests. Your initial belief was correct. When I created an empty new project, I got the behaviour that you suggested - integration tests only run with test-app and not within STS.
        Hide
        Kris De Volder (c) added a comment -

        Ok Jean, thanks for following up. Closing this ticket.

        Show
        Kris De Volder (c) added a comment - Ok Jean, thanks for following up. Closing this ticket.

          People

          • Assignee:
            Kris De Volder (c)
            Reporter:
            Jean Duteau
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:
              First Response Date: