Feature request: A TestCase that automatically finds and tests PageFactory subclasses

[ababic]

Imagine a test case that magically find all of the PageFactory subclasses within a project, and for each class found, test that:

• It can successfully create an instance of that page type
• The created page doesn't error when viewed normally
• The created page doesn't error when viewed in 'preview' mode

I think this could easily be achieved by keeping a 'reference' dict that gets populated via a metaclass when subclasses are defined (a bit like what Wagtail does with wagtail.core.models.PAGE_MODEL_CLASSES)

We could ignore any factories that do not specify a model value in their Meta class. And maybe even introduce a new/optional Meta option that could be used to explicitly opt-out of testing for a specific factory class.

[danthedeckie]

We use an extended base class for that - no magic, so we end up with

class ArticlePageTests(PageTestCase):
    factory_class = ArticleFactory
    
    def test_basics(self):
        self.assertCanRenderPage()
        self.assertCanRenderAdminPageEditor()
        self.assertCanRenderPagePreview()

where PageTestCase provides assertCanRenderPage etc.

kind of thing. Which is then pretty easy to extend with a mixin, if you like:

class ArticlePageTests(BasicPageTestsMixin):
    def test_basics(self):
        self.assertCanRenderPage()
        self.assertCanRenderAdminPageEditor()
        self.assertCanRenderPagePreview()


class ArticlePageTests(BasicPageTestsMixin, PageTestCase):
    factory_class = ArticleFactory

class BlogPageTests(BasicPageTestsMixin, PageTestCase):
    factory_class = BlogPageFactory

class ProductPageTests(BasicPageTestsMixin, PageTestCase):
    factory_class = ProductPageFactory

we've actually got it doing parent stuff too for some page types, eg:

class ProductPageTests(BasicPageTestsMixin, PageTestCase):
    factory_class = ProductPageFactory
    
    def get_parent(self):
        if hasattr(self, product_group_page):
            return self.product_group_page

        self.shop_page = ShopPageFactory.create(parent=self.site.homepage)
        self.product_group_page = ProductGroupPage.create(parent=self.shop_page)
        return self.product_group_page

type of thing. and then in the assertCanCreate or wherever, it always creates using the parent=self.get_parent() kind of thing...

Having something like this in a library would be nice - maybe we'll extract it to one some time.

[ababic]

@danthedeckie We often find ourselves doing something quite similar to that, only where you have custom assertions, we just have separate tests. e.g.:

class BasicPageTypeTestsMixin:
    factory_class = None

    def setUp(self):
        self.basic_obj = self.factory_class()
        super().setUp()
    
    def test_can_render(self):
        ...
    
    def test_can_render_edit_view(self):
        ...
    
    def test_can_render_preview(self):
        ...

Certainly not difficult to do. But, with both solutions, you still need to import this and use it in the right places. Whereas, if we had some way to identify the factories of interest, we could do this once, and we'd be covered forever.

class BasicPageTests(TestCase):

    def setUpClass(cls):
        self.page_factories = get_page_factories()
    
    def test_can_render(self):
        for factory in self.page_factories:
            basic_obj = factory()
            with self.subTest(type(basic_obj)):
                ...
    
    def test_can_render_edit_view(self):
        for factory in self.page_factories:
            basic_obj = factory()
            with self.subTest(type(basic_obj)):
                ...
    
    def test_can_render_preview(self):
        for factory in self.page_factories:
            basic_obj = factory()
            with self.subTest(type(basic_obj)):
                ...