Provide a way for admins to export everything

[jonespm]

It was requested that admins should be able to get a full export of the view. This may need to be something built in as a custom admin interface. I'm not sure the effort around this or if it would be worth putting in that time to complete this task as currently devs can export CSV's right from the view on the database.

Related to this was a way for department chairs and managers to export all of the reports for employees they manage. I'm not sure how that would be implemented and/or if it would be similar to this.

Originally posted by @jonespm in #97 (comment)

Some additional implementation details, these may be subtasks.

1. Refactor the database logic currently in ExportMeetingStartLogs.get either into the models.py MeetingStartLogsView proxy or into a separate utils file.
2. Rewrite this to be less memory intensive, avoid a fetchall and instead have a loop that does a fetchmany
3. Have a method parameter for whether or not to do the filter by user. (To make it work for admins) Or maybe have it pass in the courses to filter by or a None to do no filtering.
4. Call this method from where you are defining the admin action with no restrictions as well as in the view with restrictions.

[hanzheg]

Hello @jonespm,

My partner and I are students in the EECS 481 course, and we would like to work on this ticket as part of our open-source project contribution assignment. Could you please assign this task to me?

Thank you!

[jonespm]

Sure @hanzheg this might be a little harder one but you're welcome to try it. I think my idea was that we'd add a new custom action to the Django admin for queues. Right now there's 3 options, Delete, Hard Delete and Soft Delete. You'd be able to check the ones you also want to export and it would run it through the export code and provide an export.

The ones that are a little easier are ones with the label off "Effort: 4" and/or "Good First Issue."

[jonespm]

I'm thinking maybe something like this for the refactored model method

Then you'd pass like

    response = HttpResponse(content_type='text/csv')
    response['Content-Disposition'] = f'attachment; filename="{filename}"'
    writer = csv.writer(response)
    MeetingStartLogs.export_to_csv(queues_user_is_in or None, writer)

class MeetingStartLogs(models.Model):
    @classmethod
    def export_to_csv(cls, queue_ids, writer):
        with connection.cursor() as cursor:
            if queue_ids:
                # Filter by specified queue IDs
                queue_id_placeholders = ', '.join(['%s'] * len(queue_ids))
                query = f"SELECT * FROM meeting_start_logs WHERE queue_id IN ({queue_id_placeholders})"
                params = queue_ids
            else:
                # No filtering by queue_id
                query = "SELECT * FROM meeting_start_logs"
                params = []

            # Fetch column names and write headers
            cursor.execute(query + " LIMIT 1", params)
            column_names = [desc[0] for desc in cursor.description]
            writer.writerow(column_names)

            # Stream rows in chunks
            cursor.execute(query, params)
            rows = cursor.fetchmany(1000)
            while rows:  # Continue until no more rows are fetched
                writer.writerows(rows)
                rows = cursor.fetchmany(1000)