Simulations in historical research: How to create an agent-based model of communication networks #605
Comments
|
Dear @merrygin and @maltevogl, I hope you're both well. I'll be working as editor for this lesson. In our previous communication by email last week, you asked for some initial feedback on the basis of the first draft you sent on Colab, which you also kindly sent to me in a .md file. The draft you shared is an excellent starting point for the lesson. Much of it is structured already, the theoretical grounds are clear, and much of the code is written. I would like to offer some comments and advice before you send a full submission package:
These are all initial recommendations based on my first reading of the draft, and I hope that they will be of use to you when working on your full submission. As mentioned in the message above, we will be waiting for the submission package by April. In the meantime, please feel free to write with any questions or doubts. I'm looking forward to reading your full draft, this is a very exciting lesson and I'm glad to be able to give you a hand in the process. |
|
Hello @digitalkosovski, @merrygin and @maltevogl, You can find the key files here:
You can review a preview of the lesson here: I noticed a couple things when setting this file up, which I've listed below:
Please feel free to make your subsequent edits directly to the markdown file here on Github! Thank you ✨ |
|
Thank you for setting up this preview @charlottejmc! -- Hello Jascha @merrygin and Malte @maltevogl, I've sent you both invitations to join us as Outside Collaborators here on GitHub. This will give you the Write access you'll need to edit your lesson directly. (There's no need to use the Git Pull Request system in our ph-submissions repository). Please let us know if you encounter any practical problems as you work with Agustín @digitalkosovski to shape this draft towards its Initial Edit. Charlotte and I are here to help! Best, |
|
Hi @anisa-hawes @digitalkosovski and @charlottejmc - thank you for setting everything up and providing us with feedback! We will start updating our submission in the coming days. |
|
Dear @anisa-hawes @digitalkosovski and @charlottejmc We already tried to address some of your feedback, especially that towards structural inconsistencies, the danger of too much text as well as the code not working. We tried to overhaul Part I to make the methodological introduction to simulations more approachable and less overwhelming. We now try to start at the historical interest for the subject matter and work step-by-step towards why simulations are useful for this. In order to do this, we also slightly changed the structure of the lesson compared to our initial submission. The code should also be functional now and we extended it to include some visualization elements, especially an interactable interface inside the juptyer notebook. If you'd like to test the functionality of the code, you can try it here in the original colab. There is still a lot to do, most importantly writing up Part III, cleaning up and finishing the text of Part II and reiterating on Part I, too, including all references / the bibliography as well as setting up a (external) document that systematically summarizes the model and would be a very good end point of the lesson, showing what could be done further with the model. Unfortunately, all this is likely a bit too much for us to finish by end of April next week. So if it wouldn't be too much of an ask, we would kindly request if we might take a couple more weeks for getting everything ready. All best, |
|
Dear @merrygin and @maltevogl , thanks for keeping us posted. It all sounds good and I'm confident about your coming submission. As for your request, you can indeed take some extra days/weeks to finish the draft. Keep up the good work! |
|
Dear all, |
|
Thank you for the update Jascha @merrygin and Malte @maltevogl! I'm just tagging Agustín @digitalkosovski here so that everyone is in the loop. Please let @charlottejmc or I know if you need any practical assistance. We're here to help |
Thanks a lot! The main objective left is to make everything neater (but still short enough) language-wise and to get all the references in. Sadly, it didn't work out entirely today due to a very stubborn sickness, sorry for that! But we are actually almost done, should be finished by tomorrow or Tuesday at latest. I hope this is still fine for everybody! Have a nice remainder of the weekend! |
|
Dear @digitalkosovski , @charlottejmc and @anisa-hawes I've pushed our finished version now, sorry again for the delays! Hope you are all doing well! |
|
Oh, also, we used a certain way of inserting references for now, without knowing if this is the preferred one for you. Of course, we'll adapt that to whatever you like us to! Same goes for the actual reference style, which we have not standardized yet, for the same reason. |
|
Hello Jascha @merrygin, Malte @maltevogl, Aleksandra and Raphael
Thank you for your work on this submission! I've made some adjustments to the file:
We use The Chicago Manual of Style, 17th Edition Notes and Bibliography system for endnotes. We insert
You can review the preview here: https://programminghistorian.github.io/ph-submissions/en/drafts/originals/agent-based-model-communication-networks. If you're happy with the preview now, I think we are ready to hand this on to @digitalkosovski for the Initial Edit |
|
Dear @anisa-hawes thanks a lot for these adjustments! The preview looks really amazing already. For the missing citations I have to refer to @merrygin . The in-text references were meant as footnotes for the paragraph text. If that format does not work, maybe they could be converted to endnotes similar to the treatment of citations and referenced with e.g. [^a], [^b] to distinguish from citations? In Paragraph # 60 there seems to be a formatting error? It starts with ```python id=”tQl5YRxWeonN” “"” and the fourth level header is not picked up. Other then that it looks great! |
|
Thank you, @maltevogl. I've removed that cell id I'm unclear what you mean by footnotes for the paragraph text. We can add in an endnote to provide a note at those two lines, just let us know which works you intended to reference Jascha @merrygin? I'm confident that the header is display correctly. The typography in our Previews is slightly different than in our published lessons, but you can see how |
|
Hello Jascha @merrygin, Malte @maltevogl, Aleksandra @Aleks-Kaye, and Raphael @raphschlatt. What's happening now?Your lesson has been moved to the next phase of our workflow which is Phase 2: Initial Edit. In this phase, your editor Agustín @digitalkosovski will read your lesson, and provide some initial feedback. Agustín will post feedback and suggestions as a comment in this issue, so that you can revise your draft in the following phase (Phase 3: Revision 1). %%{init: { 'logLevel': 'debug', 'theme': 'dark', 'themeVariables': {
'cScale0': '#444444', 'cScaleLabel0': '#ffffff',
'cScale1': '#882b4f', 'cScaleLabel1': '#ffffff',
'cScale2': '#444444', 'cScaleLabel2': '#ffffff'
} } }%%
timeline
Section Phase 1 <br> Submission
Who worked on this? : Publishing Assistant (@charlottejmc)
All Phase 1 tasks completed? : Yes
Section Phase 2 <br> Initial Edit
Who's working on this? : Editor (@digitalkosovski)
Expected completion date? : July 6
Section Phase 3 <br> Revision 1
Who's responsible? : Authors (@@merrygin, @maltevogl, @Aleks-Kaye, and @raphschlatt)
Expected timeframe? : ~30 days after feedback is received
|
Hi @anisa-hawes , Regarding the two in-text references at lines 248 / 261 and 397 / 410: |
|
Dear @maltevogl and @merrygin, Many thanks for sending the full draft. After having a quick look at it, I have the impression that you've significantly improved the previous draft (which was very good to begin with), and I expect that there won't be much work to do at this point. Given that the summer break is approaching, I think we should start the external reviewing process as soon as possible. So my intention is to come back to you next week with a couple of remarks and suggestions so we can speed up the process. Hopefully we can do phases 2 and 3 in just a couple of weeks and launch the external reviews in July already. Best, |
|
Dear @maltevogl and @merrygin I've gone through the lesson, and as I expected, I think you've done great work and there are very few suggestions I have before we go to external review. The lesson is very clear and friendly in spite of the complexity of the subject. You’ve added great references, including some to other PH lessons, and I also appreciate that you’ve added some thoughts and external links concerning the data and the research but in a very efficient way, providing readers with interesting and relevant information, but without overloading the lesson itself with heavy research content. I just have a few observations and suggestions:
Maybe @anisa-hawes can confirm? As you can see, these are all very minor observations, but I think it could be useful to address these things before sending the lesson for external review. If you agree with these observations, I hope you can work on them in the next couple of weeks and send a new version by late June or early July. That will allow us to do the reviews over the summer and contact you again by the end of the summer break. I hope all of this helps. The lesson is fascinating and to my understanding, extremely useful. I'm very much looking forward to seeing it online. All the best, |
|
Dear @anisa-hawes , @digitalkosovski , and @charlottejmc , We have some thoughts on I) though: Expanding our definition of simulations with a more detailed contextualization of the term is definitely a good idea, but we have to stress that there is ambiguity in how the terms "simulation" and "model" are used in different fields and among researchers, which may not have become entirely clear from the review. So there are many definitions and classifications that concentrate on different aspects of those methods, but generally, there is no established and strict epistemological delineation of different types of simulation models that researchers agree upon. For ABMs, the much more common characterization is that it is a simulation model, and that simulation models are not limited to physical or chemical subjects, since the qualitative and epistemological difference between simulating social phenomena vs. simulating, e.g., chemical phenomena (i.e., the subject) does require a different approach to the method, but does not mean it is a different method altogether, as they share many if not all key characteristics in their methodology. Regarding the historical argument that "simulation" used to be an exclusively natural sciences term, this is actually not quite accurate, as the use of the term "simulation" describing virtual experiments of social phenomena has been going on since the first of such applications, roughly since the 1950s. I'd love to go into more detail on this subject and give more detailed arguments, but I just wanted to briefly highlight our position on that. For the reason alone that it is by far the more common term to describe ABMs in the literature, we actually doubt that many readers would be very confused by us characterizing ABMs as simulations, on the contrary. But since a more thorough explanation and argument about this will certainly help the readers navigate the multitude of definitions and viewpoints there are about this, it is still a very very good thing to do! At the very least, this new paragraph could show that there is a lot of methodological and epistemological thought and work left to do for using ABMs in history. We'll meet up soon to discuss how fast we can apply those recommendations and will get back to you on that as soon as possible. |
|
Hello Jascha @merrygin, Malte @maltevogl, Aleksandra @Aleks-Kaye, and Raphael @raphschlatt. What's happening now?Your lesson has been moved to the next phase of our workflow which is Phase 5: Revision 2. This phase is an opportunity for you to revise your draft in response to the peer reviewers' feedback. Agustín @digitalkosovski has summarised their suggestions, but feel free to ask questions if you are unsure. Thank you for sharing your thoughts on the conceptual comments Jascha @merrygin - feel free to continue the conversation/ask further questions here in the Issue thread. Again, you can make revisions via direct commits to your file: /en/drafts/originals/agent-based-model-communication-networks.md. @charlottejmc and I are here to help if you encounter any difficulties. When you and Agustín are all happy with the revised draft, the Managing Editor @hawc2 will read it through before we move forward to Phase 6: Sustainability + Accessibility. %%{init: { 'logLevel': 'debug', 'theme': 'dark', 'themeVariables': {
'cScale0': '#444444', 'cScaleLabel0': '#ffffff',
'cScale1': '#882b4f', 'cScaleLabel1': '#ffffff',
'cScale2': '#444444', 'cScaleLabel2': '#ffffff'
} } }%%
timeline
Section Phase 4 <br> Open Peer Review
Who worked on this? : Reviewers (@LeifSch + @hluling)
All Phase 4 tasks completed? : Yes
Section Phase 5 <br> Revision 2
Who's working on this? : Authors (@merrygin, @maltevogl, @Aleks-Kaye, @raphschlatt)
Expected completion date? : early November
Section Phase 6 <br> Sustainability + Accessibility
Who's responsible? : Publishing Team
Expected timeframe? : ~21 days
|
|
Dear @anisa-hawes , @digitalkosovski , and @charlottejmc , We just committed changes for the Specific Comments by Luling, a paragraph on why we chose the term "simulation" and surrounding ambiguities, a paragraph on how ABM is suited for our case study (ie., which hypotheses we could test), and some paragraphs on validation methods for ABMs. In the last one, there are some citations missing, which we will have until the end of the week, I'm sure. Also, the citation numbers still need to change due to some of the new citations... I'll message again once we have everything ready! |
|
Thank you for this update, Jascha @merrygin ! When you've gathered the additional citations, Charlotte and I will be check/adjust the numbering for you. This is something we always review as part of our Sustainability and Accessibility checks |
|
Dear @anisa-hawes , @digitalkosovski , and @charlottejmc , Thanks a lot for checking and adjusting the references that's a big help! :) If we haven't missed anything, I think everything regarding the review and suggested changes should be implemented now. All best and already wishing you a great weekend |
|
Hello Jascha @merrygin. Thank you for all your work on this! As I said, @charlottejmc and I will adjust the numbering of references as part of our Phase 6 checks. That's no problem -- Hello Agustín @digitalkosovski, This lesson is ready for your read-through ✨ Please let me know when you're happy for us to hand over to Alex who will re-read and advise if any further revisions are required before we move onwards to Phase 6: Sustainability + Accessibility. Thanks all! |
|
Dear @maltevogl @merrygin @Aleks-Kaye and @raphschlatt, Thank you so much for this new version, which is extremely interesting and I think reads very well in spite of its complexity. I also had no problem running the code, which I think you present very clearly. And I really liked your comparison of simulation with the hermeutic circle, I think this speaks to historians and humanists very well. Some very minor comments: a) line 32: There's an extra comma between notes 1 and 2. b) line 87, the phrase "it is hard to understand why and how these networks came to the shape we observe today". To me, that "today" sounds as if we were talking about present networks, not of past networks. I would also change the "it is hard to understand", because you're precisely showing that we can in fact understand them! I suggest rewriting the phrase bit and change it to something like: "The effect of this network on the history of science in Europe and the world is deemed to be pivotal. To understand these networks, it is not enough to study their shape and speculate about the historical sources we have about them. It is also essential to ask ourselves how exactly these networks came to be shaped as they were". Do it you deem best, this is a matter of style that can be settled later. c) Line 150: I would not call this a chapter, but rather a section. That is all. After you make these very minor changes, Alex can go through the lesson and we can move to Phase 6. Again, thank you all for your hard work. I'm looking forward to seeing this lesson published soon. All the best, |
|
Dear @digitalkosovski Thanks a lot for this positive feedback and the additional suggestions - I just implemented them as is. So from our side, I think we are ready for the next Phase. :) And thanks to you all once again, too, for all of your hard work in ensuring that our submission is sufficiently polished! I think all the feedback helped it a great deal. All best, |
|
Thank you, Jascha @merrygin, Malte @maltevogl, Aleksandra @Aleks-Kaye, and Raphael @raphschlatt, Next, the Managing Editor Alex @hawc2 will read your lesson to confirm if he feels it is ready for Phase 6, or if he'd like to suggest any further revisions before we move onwards. Anisa |
|
@merrygin @maltevogl this lesson looks very well put together as is, and my feedback for revision is minimal. Your lesson covers in clear language an important and complex topic, and fills a gap for PH. With that said, it does strike me as a difficult lesson, and we should probably label it as Advanced rather than Intermediate. This type of modeling is a difficult, and somewhat esoteric method, even if it has wide applicability, and it may be helpful for the reader if you can say a bit more about the disciplinary contexts in which it's commonly used, providing more examples of scholars or research projects that utilize it effectively for a range of different questions. You cover this in a sense in section 1.4 but some important details are buried. Was this method developed in the field of digital history, or another social science? And what do you mean by Digital History? These terms may not be so generally understood or accepted. For instance, shortly after suggesting it orginated in digital history, you write: "Agent-based Modeling as a term for the kind of simulation approach we just described was coined during the 1990s, pioneered among others by Joshua M. Epstein and Robert Axtell14", you don't elaborate on who those scholars are, and what their relationship to broader academic disciplines are. This doesn't require alot more info, but this sort of brief context will help readers understand the origins of this methodology and how it can be applied in their field. My only other suggestion for revision is to add more about Prerequisites necessary for this lesson, preferably before you discuss Technical Requirements. You can reference other, more introductory Programming Historian lessons on Python and related methodologies. But you should provide at least a couple paragraphs where you situate for the reader the difficulty level of this lesson, what the lesson assumes they already know. This is an opportunity ti also make clear for the reader here what gaps in PH's lesson repertoire this lesson fills, how it relates to and builds on other lessons but also goes in a new direction. You can also gesture here towards the competencies reader will develop by following along with the lesson. I'd recommend considering at a few places later in the lesson that you remind the reader of certain prerequisites or relevant Python skills they are learning. When the lesson gets into the details of running the code, it can be helpful to provide more signposts and guidance for readers who might get lost in the weeds of the code. It's not required but this is your last chance to make any substantial revisions to make the lesson more legible to those who don't have a ton of Python or programming experience. Once you make these revisions, I'll do a light line edit and let you know if any last questions arise. Then I'll hand it to Charlotte for a final round of copyedits and prep for publication. |
|
P.S. one way to provide clarity about the context for using this method would be to lean less on your endnotes and provide more substantial info about each source you reference. You have some substantial references in endnotes that aren't really described in the main body of the lesson in any detail. In many of those cases, you could afford to add another sentence describing the source you are citing and clarifying how it is relevant, especially when it helps you situate this methodology for its usefulness to a specific disciplinary research question. |
|
Dear @hawc2 , |
|
Thank you, Jascha @merrygin. Please write to us if you have questions in the meantime. |
|
Dear @hawc2 and Dear @anisa-hawes , @maltevogl and I finally got around to implement the desired changes. We took the opportunity to pass around the text to @raphschlatt and @Aleks-Kaye once again, especially with the last feedback in mind. We especially tried to extend some parts for clearer disciplinary contextualization, with explanatory sentences, clarifications, and further references. We also made efforts to make visualizations clearer/more effective in communicating what is shown. Regarding the question of "Digital History" as a label for different approaches, especially the history of the method, we thought it might be best to not include a whole new section on this discussion. We did rephrase and extend many parts in the sections on Agent-based modeling and its history, as we felt that perhaps they were a bit misleading. ABMs did not emerge from historical simulations, but in other fields, and are only now adopted into modern "Digital History" and Digital Humanities. The examples of historical simulations in the 70s were just illustrations of how these types of method (not exactly that method, ABM) were around for a long time, and are not entirely a new thing for computer-based historical inquiry (while always having been niche, of course). I hope these sections are clearer now and might also alleviate the need for a more in-depth discussion of the term Digital History! Otherwise, we also referenced new literature that leads to that specific discussion. We also extended the prerequisite section to make more clear which concepts / functions of python are necessary to follow the lesson. We did not include much more than that in the coding parts of the lesson (see the matter of signposting) - we felt that this might be distracting or also a bit superfluous in the cases were we considered and discussed it. But, just to reiterate, we tried to be as extensive as possible in the introductory Technical Requirements section, linking to other resources (including on PH) if readers might feel they need more background. We hope these decisions and extensions are satisfactory to you! One other question that crept up during our team review was regarding the code snippets. One of our team members felt it might be confusing to present some code bits, which are just examples of elements of the model, and other bits, which are proper, executable parts of the lesson, in the same way. We were unsure how to deal with that - if at all - and just wanted to mention it here. Maybe a different formatting of those bits might suffice? Or maybe you have the experience that this is not a big issue for most readers? One example of such a case is in section 2.5.5 Adding Space, where the first two code blocks (starting with self.grid) are just for illustration purposes and only the third one is to be executed. Thanks again for all the hard work and extensive feedback rounds! We are looking forward to the next steps! All best, |
|
Thanks @merrygin for your revisions! This satisfies most of my concerns, and I agree the important point to clarify is that: "ABMs did not emerge from historical simulations, but in other fields, and are only now adopted into modern "Digital History" and Digital Humanities." One place where this could still be clarified is in the section entitled Historical Context of Agent-Based Modeling. It's hard to follow in those first paragraphs which research happened when. In the first paragraph, you don't give any dates. When did the pioneering figures like Levison do their research? It's hard to follow because in the next paragraph you make reference to innovations in the 1990s from other fields, but it's hard to tell if you're saying the pioneers in DH came before or after that. This whole section almost goes backwards in time, starting in the 90s, then going back to 50s and 30s. Could it be reversed, so the history of this research is presented more linearly in chronological order? For prerequisites for Python, I think what you've done is sufficient. The only thing I'd ask is could we cite a couple relevant PH lessons that could serve as prerequisites to yours? For the code snippets that you don't intend the reader to use, that's a more complex question. Which code blocks are these? @anisa-hawes do you have thoughts on this issue? |
|
Thank you for all your work on these revisions, Jascha @merrygin. In my opinion, breaking down the code in the way that you have with thoughtful insight and commentary is great. The process of developing an effective methodology isn't linear, and necessarily involves iterative modification, learnings, and revisions.... I think you communicate this well. My view is that your readers will have an opportunity to handle your code and learn from it with your guidance - that's a very good thing. |
|
Dear @hawc2 , The issue of the historical timeline and missing dates is a very good point. I restructured that part again so it starts with a paragraph on ABM and its roots in general, and then the rest is more chronologically ordered going from early historical simulations, to more modern ABM approaches in comparison, then towards recent applications. Regarding linking to PH lessons in the "Prerequisites" section, we already do link to the start of the Python lesson-series ("Python introduction and installation") as well as the "Introduction to Jupyter Notebooks". Would you say that suffices there, or would you prefer some more linkages? In later sections we also do link to more specialized tutorials on PH that fitted the current objectives (Visualising Data with Bokeh, Understanding and Creating Word Embeddings, Understanding and Using Common Similarity Measures for Text Analysis). Thanks again and have a great remainder of the week! |
|
Dear all, I just wanted to thank you all for these exchanges, which I think will have a very positive impact on the lesson's final version. I think Alex had a good point with his suggestion for a clearer narrative in the section about historical context. I'll let him judge the last changes, but I feel that things are going in the right direction here. All the best, |
|
Thanks @digitalkosovski, and thanks @merrygin for your edits. The history section looks good now. I missed those ProgHist lessons you cite simply because they aren't labeled as such ("for more on Jupyter notebooks, see as a prerequisite the Programming Historian lesson, "Introduction to Jupyter Notebooks"). But that's the kind of edits we can do during copyediting, mostly to make sure links are explicit with titles to the corresponding articles. With that, @anisa-hawes @charlottejmc, I think this is ready for copyediting! |
|
Hello @merrygin and @maltevogl, This lesson is now with me for copyediting. I aim to complete the work by ~28 November. Please note that you won't have direct access to make further edits to your files during this phase. Any further revisions can be discussed with your editor @digitalkosovski after copyedits are complete. Thank you for your understanding. |
charlottejmc
moved this from 5 Revision 2
to 6 Sustainability + Accessibility
in Active Lessons
|
Hello Jascha @merrygin, Malte @maltevogl, Aleksandra @Aleks-Kaye, and Raphael @raphschlatt, What's happening now?Your lesson has been moved to the next phase of our workflow which is Phase 6: Sustainability + Accessibility. In this phase, our publishing team will coordinate a series of tasks including: copyediting, typesetting, generating archival links, collating copyright agreements, and reviewing essential metadata.
When our Sustainability + Accessibility actions are complete, the Managing Editor @hawc2 will read the lesson/translation through one final time ahead of publication. %%{init: { 'logLevel': 'debug', 'theme': 'dark', 'themeVariables': {
'cScale0': '#444444', 'cScaleLabel0': '#ffffff',
'cScale1': '#882b4f', 'cScaleLabel1': '#ffffff',
'cScale2': '#444444', 'cScaleLabel2': '#ffffff'
} } }%%
timeline
Section Phase 5 <br> Revision 2
Who worked on this? : Authors (@merrygin, @maltevogl, @Aleks-Kaye, @raphschlatt)
All Phase 5 tasks completed? : Yes
Section Phase 6 <br> Sustainability + Accessibility
Who's working on this? : Publishing Team
Expected completion date? : 28 Nov
Section Phase 7 <br> Publication
Who's responsible? : Managing Editor @hawc2
Expected timeframe? : ~10 days
|
|
Hello Jascha @merrygin, Malte @maltevogl, Aleksandra @Aleks-Kaye, Raphael @raphschlatt, and @digitalkosovski, I've prepared a PR with the copyedits for your review. There, you'll be able to review the 'rich-diff' to see my edits in detail. You'll also find brief instructions for how to reply to any questions or comments which came up during the copyedit. When you're all happy, we can merge in the PR. |
Programming Historian in English has received a proposal for a lesson, "Simulations in historical research: How to create an agent-based model of communication networks" by @merrygin and @maltevogl.
I have circulated this proposal for feedback within the English team. We have considered this proposal for:
We are pleased to have invited @merrygin and @maltevogl to develop this Proposal into a Submission to be developed under the guidance of @digitalkosovski as editor.
The Submission package should include:
We ask @merrygin and @maltevogl to share their Submission package with our Publishing team by email, copying in @digitalkosovski.
We've agreed a submission date of April. We ask @merrygin and @maltevogl to contact us if they need to revise this deadline.
When the Submission package is received, our Publishing team will process the new lesson materials, and prepare a Preview of the initial draft. They will post a comment in this Issue to provide the locations of all key files, as well as a link to the Preview where contributors can read the lesson as the draft progresses.
_If we have not received the Submission package by April, @digitalkosovski will attempt to contact @merrygin and @maltevogl. If we do not receive any update, this Issue will be closed.
Our dedicated Ombudspersons are Ian Milligan (English), Silvia Gutiérrez De la Torre (español), Hélène Huet (français), and Luis Ferla (português) Please feel free to contact them at any time if you have concerns that you would like addressed by an impartial observer. Contacting the ombudspersons will have no impact on the outcome of any peer review.
The text was updated successfully, but these errors were encountered: