Webinar Materials

On October 3, 2023, Stacey Arnold from the National Center for Biotechnology Information (NCBI) ClinicalTrials.gov team gave a brief overview of the modernization of the PRS, the system used to submit clinical trial information that is posted on ClinicalTrials.gov. She provides a mini tutorial on how to register a trial using the modernized interface, PRS Beta. Following the presentation, NLM product staff from the NCBI ClinicalTrials.gov team answered questions from the audience.

Links

• ClinicalTrials.gov

• Drive-thru Data: API

• Unified Medical Language System (UMLS)

• NLM Technical Bulletin

• NLM Support Center

Transcript

LOUISE: Welcome everyone to NLM Office Hours. My name is Louise To. I am a Technical Information Specialist from the Office of Engagement and Training at the National Library of Medicine (NLM), and I'm going to be your moderator for today's Office Hour. If this is your first time, NLM Office Hours is a chance to learn more about NLM products and services and get your questions answered. To provide broader and continuing access to the information, these sessions are recorded and then later posted on NLM's website.

NLM Office Hours is brought to you by the Network of the National Library of Medicine (NNLM), which is the education and outreach arm of the NLM, and they work to advance the health and wellbeing of everyone through improved health, information literacy and information access. NNLM works regionally, helping organizations within communities serve the public they know best and nationally, offering online educational programs. Feel free to visit NNLM.gov to learn more.

So today for NLM Office Hours, we have Stacey Arnold from the ClinicalTrials.gov PRS team, also known as the Protocol Registration and Results System. Stacey will give an overview of the modernization of the PRS, the system used to submit clinical trial information that is posted on ClinicalTrials.gov. She will provide something like a mini-tutorial on how to register a trial using the modernized interface, the PRS Beta. Assisting me in managing today's session is Michael Tahmasian and Kate Majewski, also from the Office of Engagement and Training at NLM.

You will have three ways to communicate with us during today's session. You can use the chat feature and we'll be watching for your questions in the chat. You may also raise your hand if you'd like to unmute and ask questions verbally. I will be looking for those hands. And you may also use the feedback icons to give reactions throughout the session and that's located on the bottom of your menu options in Zoom. Also, a note that we have a closed captioner with us today. You can find the closed captioning button in your menu options. And now without further ado, let me pass this on to Stacey to tell us about ClinicalTrials.gov PRS.

STACEY: Thanks Louise. I'm going to start sharing my screen now and hopefully it will go seamlessly. OK, hopefully everyone can see my screen, and let me actually go back a sec. OK.

So thanks everyone. As Louise mentioned, my name is Stacey Arnold. I am a Protocol Registration and Results (PRS) database subject matter expert with expertise in results and general system functions. I'll be describing our efforts to modernize the PRS and I'll also demonstrate the current features of PRS Beta. So to begin with, I'll give a brief overview of the progress of the modernization effort so that we can spend the majority of our time looking at the beta website.

Over the past few years, we've been working to modernize both the ClinicalTrials.gov public website and the PRS database, as Louise said, which is the database that is used to submit records for publication to the public website. In Year 1 of this process, we engaged stakeholders to determine needs and plan our approach to modernization of both websites. We enhanced our internal processes and developed roadmaps. And then in Year 2, we launched the first release of the ClinicalTrials.gov Beta. In Year 3, we launched the first release of PRS Beta and we've been conducting usability research and making iterative improvements since those first releases. As we continue modernization in Years 4 and 5, we made ClinicalTrials.gov Beta primary and we'll continue to engage stakeholders and make refinements in response to feedback as we strive to make the PRS Beta primary as well.

This is a view of the road map for PRS Beta and the work that remains to be completed. You'll see that we've been working this year to complete the Protocol section and to allow for submission and QA/QC review of registrations. As we head into 2024, we'll be working on adding the Results’ and Documents’ sections and the corresponding processes for results submission and review. Finally, we'll focus on supporting account management from within PRS Beta.

In the next few slides, I'll describe how to access PRS Beta. To access PRS Beta, users must first log into the classic website. When they arrive at the Record List, they will see a big, green button that says “New PRS Beta Homepage”. Clicking on this button will take users to the PRS Beta Record List and users can return to the classic website at any time by clicking on the “Back to Classic” website link at the top of the page. You'll also see there's a Feedback button on this page and you can use that from any page to provide us with feedback about what you're experiencing as you play around with the beta system.

When we talk about the classic PRS website, we are referring to not only the live PRS which we have been calling the PRS production website, but also the PRS Test website. Users can access PRS Beta from within either website, and for those who haven't used the PRS Test website, anyone can get an account here. Some of you may already have one if you had a PRS account before 2016.  And it is a playground version of the PRS production system where you can go to practice data entry before submitting records in the live system. So if you happen to have a PRS account before 2016 and you didn't know you had a PRS Test account, you use the same login information that you had been using in 2016 or before.

So we've been utilizing the test system to ready PRS Beta for release to the live PRS production system. Updates to PRS Beta are first released in the PRS Test system because this allows us to test in a stable environment and detect and fix bugs before we deploy PRS Beta to the production system. Any users with a PRS Test account can provide feedback, and the system supports both administrator and normal user accounts. Once updates are released to the PRS production system, we continue to collect feedback to fix any additional bugs that haven't been detected previously. And records that are created in PRS Beta and updates that are made to existing records in PRS Beta can be accessed in the classic environment in either the PRS production or PRS Test System.

In the remainder of the presentation, I'll give overviews of the various sections of PRS Beta that we've released so far, and I'll begin with the Record List. So this is a view of the top menu of PRS Beta as accessed from the Record List page. This top menu is available on any of the pages within Beta. So if you click on the Records menu, you can return from an individual record to the Record List at any time, and you can also create a new record. From the About menu you can access the About PRS Beta document, which describes how to navigate the Record List and provides some troubleshooting tips, and you can also access the Release Notes for every PRS Beta released so far. Finally, if you click on your account details to the far right, you can access your profile information, the list of administrators for your organization, and you can also log out of the system this way.

So all columns that are available at your level of user access are now conveniently provided in a single location. You can click on the Customize Columns tab to pull up a modal that lists all available columns. You can check or uncheck the boxes to the left of each header to add or remove columns from the view. You can drag and drop columns to change their order or move them with the up or down arrows. And you can hover over information icons to learn more about the type of data that each column provides. Note that if there is a column of particular interest- we've been getting questions about the Problems column- this modal can be used to move that column to the top of the list so that it is always present in the viewable frame. So you can actually filter for information in the column that way. And I think that we are looking to also maybe create a button that allows people to see their problem records in a more easy way. So you can see on this view that Feedback button as well.

So the reorganized column view can be named and saved by clicking on the Saved Views column button, and this button can also be used by administrators to select standard selections of columns that include the Planning View or the Public Website View. The Export button can be used to export the current view or all of the available columns, and each column can be sorted and filtered, and links to the left of each listed record can be used to open the record in either the classic system or in the Beta system. In addition, there are links within each row that can be used to email staff, and some rows also contain more information about select columns, such as the FDAAA status or columns that indicate when updates are expected.

So in summary, from the PRS Beta Record List, users can now reorder columns, access all available columns from a single location, save views, filter data in all columns and not just a select few, and send emails directly to staff members who are responsible for each record.

So now we'll move on to the Protocol section. If a user clicks on “Create New Record” in the Records menu, they will see the following page with tips for creating a record that are presented in easier to understand language. Next to each data element name, users will see an information icon. Clicking on the icon will pull up drawer content that includes a brief description of the data element and tabs that, when expanded, provide additional information and the data element definition. Users can directly access the relevant data element definition now for the data element of interest without having to scroll through definitions for the entire module.

And here we're highlighting the Secondary ID data element to show some important improvements in data entry. On-screen content has been added to emphasize the importance of including information for relevant NIH-funded grants in this section. More than one type of Secondary ID can be selected for inclusion at one time, and more than one of each type of Secondary ID can be added on the same page. For comparison, in the classic system, the Secondary ID button pulls up a menu from which only one option can be selected at a time, and the user must continue to click and select from this pop-up to add as many Secondary IDs as are needed.

Once a record is created, the user is directed to the Protocol Summary and can use a side navigation panel to access the remaining modules for inclusion of additional registration information in any order. At the top of the page is a notice that provides important information about potential differences between PRS Beta and the classic system, as well as the Brief Title, NCT Number, and Unique Protocol ID for the record currently being edited for ease of reference. Once all of the protocol information has been included, the user can navigate to the Record Summary tab to proceed with reviewing the record and submitting it for PRS QA/QC review.

So in summary, enhancements to the Protocol section include more immediate access to information about each data element. Essential information is provided on-screen and additional information is provided in a pull-out drawer where individual components can be expanded or collapsed for ease of reading and the relevant data element definition is readily available. The information that is provided is now easier to understand. There's also improved navigation and information for each module can be added in any order. There are also fewer clicks required to provide relevant details. The website also has a modernized look and feel.

In the final slides, we'll take a look at the Record Summary page. At the top of each page within a record, are a couple of menus that allow users to contact staff and perform additional record actions. If you click on the People menu, you can find out information about the Record Owner and the Last Updater, as well as update the list of staff with access to the record. If you click on the Record Access option, you will see that you can now search for users and email users directly from the list. If you click on the Actions menu, you can change upload options and download, delete, or copy the record. If you click on the Download Record option, you'll see that multiple download options can be selected at once and that content has been added to explain the purpose of each of the download options.

The Record Summary page also includes dates and other information that have been reorganized for ease of review. The Record Status bar is still available on the page and indicates the current state of the record, and information icons are used here as well to link to drawer content that describes the kind of information that might be displayed for the field that is interactive for certain fields. So, for example, the data elements that contribute to the determination of the FDAAA status are listed for each record.

Farther down on the Record Summary page, validation messages are listed and summarized in an easy-to-read panel. You can click on an individual module in the list to the right to fix the messages associated with that module. Once you've navigated to the module, you'll see that the messages are listed on the top of the page and you can click on each accordion to expand. You can also scroll down to the relevant data element to see the relevant messages listed in line with the entries that need attention.

When all validations have been addressed and you return to the Record Summary page, you'll see a message in yellow that explains that you'll need to return to the classic system for the present time to submit the record for review. This message also clarifies that some additional revisions might be required as we're finalizing the implementation of the validation messaging in PRS Beta. In the short term, users will click on the blue “Complete Next Steps in the Classic PRS” button to submit the record in the classic system.

So in summary, enhancements to the Record Summary page include more accessible content with added functionality, improved messaging and navigation to fix errors and address warnings, and reorganized content to facilitate user review.

So as we continue modernizing the PRS, we'll be adding the QA/QC process for Protocol Registration, the Results section, and the QA/QC process for results submission. We'll also continue to rely on user feedback to find out what is and isn't working for everyone and we'll continue to make improvements based on this feedback.

So with that, I'll say thank you for your attention. And now we can move on to answering questions with the help from our expert panel, which includes our Product Owner, Nachiket Dharker, our subject matter expert with former user expertise, Maureen Strange, and our Technical Lead, Ben Babics. Thank you.

LOUISE: Thanks so much, Stacey. So we do have a hand raised. I think so. From MSMOLLOY.  I guess, well, I recognize your name from last week's Office Hour and we did pass along your questions to the ClinicalTrials.gov team. But I want to give you the opportunity now to unmute yourself and prompt your questions. And I'll ask, since you have several questions, if you can do them one by one, different subject matter experts are able to answer your various questions. So please go ahead.

MSMOLLOY: Yes, I guess I'll start off with the last question I posted, Will the modernized website have an audit trail? At our institution, we often experience PIs going and changing their Primary Completion Date or Study Completion Date to 2-3 months in the past. And so it makes it really hard to track when they make these changes because it doesn't always notify us, or they just do it at the last minute. So if we can see an audit trail that would be really helpful. Just so we can track to see who's doing what and how many are out of compliance with time points for CT.gov.

NACHIKET: I can take that one. I think that's a great question. So you know, I don't think we, at least at this point, we are planning for to do anything different than what the classic system is doing. That means there are all the versions that are released, you can audit them. So, you know, between every release, you can  track what has changed, but within a version that is in progress, let's say, you know, if you-- I think what you are trying to ask is before a record version is released and people are making updates, is there a way to track them? And that is very difficult from the systems perspective right now for us to do, especially because we know that we anyways have to, you know, track and provide links to the versions that have been released and reviewed and, you know, that have comments. So we are planning to continue supporting that, but within a release at this time we are not planning on doing - or with in-progress versions, you know, if there are multiple updates made on let's say several days, within several days, there is no way for us to track it at this point.

MSMOLLOY: Thank you. I guess my next question would be the PRS Test website doesn't allow you to fully even input a record in the Beta version. Will that become available prior to the classic website being inactivated? And with that, will we receive a Training the Trainer prior to classic website being deactivated?

STACEY: So I can answer that one, if it's okay, Nachiket. So yeah, I mean, we're not going to launch PRS as primary until it's fully functional. And so, the classic system will not go away until we know that the PRS Beta, once it becomes primary is functioning as expected. So, you know, it's really important as we continue to release that we get feedback and we know when things are kind of happening that are unexpected, because we do try to control for everything but, you know, there are things that kind of surprise us at times. And so we're hearing from folks and it's been great. And then I think, -sorry, can you repeat the second part of your question?

MSMOLLOY: So, there's usually a Training the Trainer training that comes out, right?

STACEY: Okay. Yeah. So actually we're in the process of planning another Train the Trainer. But as you know, we have sort of limited availability at any time for the Train the Trainer. We, I think, typically have about 30 people in attendance. And so we typically also have maybe one person from each representing institution or represented institution, but we are planning another Train the Trainer. I don't know right now what the plan is for incorporating any information about the Beta for that Train the Trainer. But as we're moving forward, you may be familiar that we have a training resource called the PRS Guided Tutorials that are associated with the classic system. And we do plan to make updates so that those are updated to direct people through data entry for the PRS Beta. So we'll have that resource. We'll be updating other resources that are linked to entry of information into the PRS and hopefully you know in future we do plan to continue these Train the Trainer workshops. So that'll definitely be something that we'll be doing continuing into the future.

MSMOLLOY: Okay, Thank you. My final question is the classic website has a Problem Record tab, which is really useful when I log in in the morning. That's the first thing I do, is check the Problem records, make sure what has changed, what hasn't changed, so I can send out emails or verify information. The beta website, when I go to use the Problem Record column and I put it as my number two spot, doesn't accurately reflect the same amount of problem records that are in classic versus beta. And so will we have a separate tab when the Beta becomes the full website for that so we can separate out Problem records or what's the plan?

STACEY: So I think what you're describing may just be -there could be some still discrepancies between how the beta system is processing that information versus the way that the classic system is. But it could also have to do if you have more than 1000 records, you might be limited by that threshold as to which records you're seeing. And so I think that's another issue that we're figuring out - how to display all of the records for your organization so that you're not limited to a subset of your records, and you can see every record that has problems. I'm going to let Nachiket address the question of the button because you did address and you're aware of how you can reposition the column and filter that way. But I'll let Nachiket address the second part.

NACHIKET: And the second part was about like reintroducing the button. Stacey, is that—?

STACEY: Yes.

NACHIKET: OK. Yeah, I mean what we were hoping, I mean when we kind of took that button away, our hope was that we are, you know, introducing the Problem column as like a column in the Record List. So using the customized column options and repositioning that column, people will be able to, you know, have the same kind of experience as they were having with the problems record button. But if our users think that, no, that is not replicating or that functionality is not going to be, you know, totally completely replicating that problem records button then you know it would be great if you can provide that feedback, so that we can take it back. If we hear that from more users, then we'll obviously-- we want to support the users as much as possible -so we'll reintroduce that button. Again, as I said, our hope was that people won't need that button if we enhance or update our Record List in the way that we have done. But that's something that we definitely can change If that's not helping.

MSMOLLOY: Thank you so much. I appreciate it.

NACHIKET: Sure. We do recognize that it is a very important feature that our users have been using.

LOUISE: OK, thank you for the questions. We have another question from Michael. Will the PRS Test system be updated so we can practice using the new system without doing so in the live PRS production system? And it looks like maybe Stacey might be able to answer this one.

STACEY: Sure. So I reviewed, you know, that PRS Beta is available in PRS Test as well in the as in the production system. And in fact, any updates that we're making, we're testing out in the Test system first. So those who are familiar with Test realize that it's not a system where you work on your record and then put that record into the production system. They're separate systems with separate data. But it is an environment where you can test it out and make sure that it's working the way that you would expect it to before you try to submit records using it from within the production system. So hopefully that answers the question.

LOUISE: Thank you, Stacey. Our next question is from Lo. When customizing the column display, does that customization carry over to the other PRS administrators using the account? And back at you, Stacey.

STACEY: I think it's locally saved when you save your column preferences, so it wouldn't be transmitted to other admins within your organization. It's unique to your login. But you know, you could always share the information of which columns you selected for with, you know, your colleagues if you think it's useful for them to do the same kind of sorting.

NACHIKET: And I'll just jump in that I think - and add to what Stacey said. I think when we planned this, the idea behind this, you know, is that the preference of how to customize the columns is user-based rather than organization-based or, you know, that can be transferred between administrator, administrators and other users, because every user can have their own preference. So that's the thought behind that.

LOUISE: Okay. Thank you. Another question we have lined up is from Sharon Reynolds, Is there an expected date that the classic website will be deactivated? And Nachiket, you might be able to answer this one.

NACHIKET: So the short answer is no, we don't have a date. Based on what Stacey demonstrated, we have a long way to go for, you know, still with PRS Beta modernization. So you know, we have to still implement the result screens, QA/QC, and other modules or other parts of the PRS. And we can only even think of retiring classic PRS once we have all the functionalities that the classic PRS currently provides, we are able to support all those functionalities in the Beta system. And then, you know, as Stacey also mentioned, you know, we want to give people enough time to adapt to the new system and then we will think about retiring the classic system. So currently we don't have-- I cannot give you a certain date on that. It's still a while.

LOUISE: OK. Thank you, Nachiket. A question from Zuli. We have a study that currently has a record status of public. When the NCT number is searched in the public website, the study does not appear. Would you be able to provide some insight on why this may be happening or how this can be resolved? And I'm going to pass it over to Maureen to answer this question.

MAUREEN: Thanks Louise. Zuli, it's a matter of timing potentially on that. They are not automatically put on ClinicalTrials.gov the same day you may see it as public in PRS. I would recommend that you send an email to register@clinicaltrials.gov and they should be able to help you with that. And Stacey or Nachiket, if you have anything to add, please feel free.

STACEY: Nope, that would be my guess as well, that it's a timing issue.

LOUISE: Okay. Let's see, we have a comment from Carla. “The Problem button is needed. Please add it back” Thank you, Carla for the feedback. I think the team has duly noted that the button is very crucial.

NACHIKET: Yep, thanks again. Yeah, I agree. Thanks, Carla for the feedback. Although I do just want to emphasize, I hope you have tried the Problems column and how that works and then you are making, you know, providing this feedback. So first please, I would really appreciate if you try out what we have provided and if your comment stays, I'm definitely-- we are hearing this from more and more users, so we will definitely support this request if we keep hearing about this.

LOUISE: And actually, I'm sorry I've missed a question from Jessica earlier. Jessica asks, If you send an email from PRS, does it log that in the record somewhere? For example, will I be able to tell my colleague already emailed Doctor X about the PRS comments in their record? And it sounds like this is something that the ClinicalTrials.gov PRS team has to sort of consider. Nachiket, I don't know if you want to make a comment about this.

NACHIKET: Yeah, I mean in the Beta PRS we have still not, you know, reached a stage where we rethink the email and notification system. I mean we do have some, you know, early initial thoughts and plans based on again the feedback that we have received. But again, as you know, me and Stacey have said we currently are really focusing on the data entry screens, the Protocol screens, the QA/QC, the Results data entry screen. In the classic system though I think there is a limit. I mean to answer your question-- again Stacey or Maureen, feel free to jump in.

You know there are a few, you know, ways where when you-- and if you use those features then that information is saved. For example, there is a feature available, I think only to the administrators, it's called Email to Owner. And if you use that feature, then that you know can be, you know, that message can be saved or if there is a Record Log, you know, feature at the bottom of the record. And if any communication that is, you know, that our data submitters do using that Record Log that gets saved in the record. But besides that, I mean if you just email, you know using the email icons that you know that is not currently being saved in the classic system anywhere in the record. Again Stacey, Maureen, feel free to chime in.

MAUREEN: Nachiket, the only thing I can offer is also to cc your colleague on the email. You are able to add people to the email itself.

LOUISE: OK, thanks Nachiket and Maureen. Another question from Msmolloy, Will the PRS Test system ever have a point where we can receive comments from mock protocol submission to ensure we are not making mistakes when conducting self-training?

STACEY: That isn't really something we can support. I know that our registration team is really busy just with records that are submitted through the live system. We don't have anything other than validated, system-generated validation messages that the system supports to kind of warn you about things that should be fixed that the system can detect. Otherwise, it would require, you know, human intervention. And I think it's just difficult with as many records as we have coming through, to support something like that in the test system. But we would direct you to the review criteria for registration and for results. These are available on the ClinicalTrials.gov public website and I think also from within the PRS Beta-- or sorry, from within the PRS. So those would be ways to get a heads-up. You can also find some descriptions of common errors in the PRS Guided Tutorials to try to head off some problems that you may face when you're trying to register or submit results for your record.

LOUISE: Thanks Stacey. And a follow-up question to an earlier question asking about when the classic website will be deactivated. This one was asking, If there is no classic website deactivation date at the moment, is the 2024 year timeline still accurate? And I'll pass that to you, Nachiket.

NACHIKET: Yeah, I can take that. So good question. So the timeline for 2024, that's the timeline that we are aiming to make PRS Beta our primary website. So that means when you log into, you know, the classic system you will be or you log into PRS, you will see the beta screens rather than, you know, seeing the classic screens as you are seeing now. However, that doesn't mean we are going to retire the classic PRS. As it is happening currently with the public website, the public website is now the modernized public website is the primary website, although there is a button that users can use to still, you know, continue using the classic website.

The same thing will happen when we, you know, make Beta primary - PRS Beta primary- that we are aiming for in like sometime in 2024. The option to go back and continue using the classic system will still be there. That link will be provided because, again even when we make Beta primary, that would be only the time when we think that at least the basic important features or functions that our users need to be able to submit protocol and results are available. But there'll be many more, you know, or several more features that we will still need to accommodate or, you know, implement in the Beta - in the new system. So again those are the two, you know, dates and I think that was the reason for this question or confusion. So there is 2024 is for Beta becoming primary and then as I said, we don't have a date for retiring the classic.

LOUISE: Thanks Nachiket. A question from Lo-- or another question from Lo, Will there be a way to view a highlighted track changes version of an updated ClinicalTrials.gov record without clicking the Approve and Release buttons first? Stacey, I wonder if this is question for you.

STACEY: I think we're currently working on generating those highlighted differences and I do think that it's still coming after the Release stage. Of course, you could always unrelease the record if you found that, you know, something didn't get updated that needed to be updated, or was updated inappropriately. So I would suggest, you know, reviewing before you hit the Approve and the Release and then, you know, maybe after just to make sure. But there is a period of time where you can grab that record back if you need to.

LOUISE: Thanks Stacey. And our next question-- and actually I want to acknowledge that Carla, we saw your second comment that in reference to the record button, I think that you've tried it and your comments stay. So we definitely hear you on that.

But for your follow-up question or your next question, Is there any way to make the columns narrower to add more to a page view so you don't have to keep scrolling to the right? Can a row be highlighted as it is easy to get off track with all the scrolling to the right? I find the modernized PRS system very cumbersome to use and it makes me spend at least twice as much time to conduct the same task as in the classic system. Maybe that's a Stacey question.

STACEY: You know, I think I might let Nachiket answer that.

NACHIKET: I mean they are different again, I think they are different. First of all, thanks for that feedback towards-- we do definitely take it and we will take it back to our team. We already have, you know, some plans to, you know,-- we did realize certainly, you know, based on the feedback that we have been seeing that we could, you know, make better use of the real estate for example and other things. But any/all of this feedback is really helping us, you know, continuously improving the system. The first part was I think or the question was I think, to do with adjusting the column width, and I think that's possible, right? So I mean you can change the column width by dragging or adjusting the columns so that you can have a better view, if that answers the question. And I think when you select a row it does change. Again, Stacey, Maureen, feel free to jump in. It does change or shade. So I mean I don't know exactly what kind of highlighting would help or are you looking for, but it does, you know, point or give that appearance to the user that that's the row that you are using. But yeah, I get the point about scrolling.

STACEY: I'm not sure about the row. I know that the column will be highlighted that is currently the focus. But yeah, I don't know about the row, individual row.

NACHIKET: We can also look into that. So we will note that feedback and we can again take that to our team too.

LOUISE: Yes. And we are seeing several other questions about scrolling from Myra and Carrie and I'm just going to note that the team is seeing them. So they duly take your comments and suggestions into consideration. So thank you Myra, Carrie, Carla for the comments on that. OK, so good.

NACHIKET: I'll just jump in because I know Carrie has provided that feedback earlier. So, Carrie, yes we did take that feedback back, you know, to our team and our design team has some solutions in mind. We are, you know, right now focusing of course on the QA/QC and other parts, but we definitely are going to think about ways how we can avoid that scroll, like scroll bar within the scroll bar. Back to you Louise.

LOUISE: Thank you. Yes. I think we're seeing a lot of hand clapping and thumbs up. So I think people are liking what they're hearing about their comments getting considered for scrolling.

OK next question is from Diane. Was the recent kerfuffle with very old records showing an error regarding Eligibility Criteria when they had entered it appropriately a decade or more ago when the record was created a function of the modernization? Can anything be done to back this concern out or be sure to avoid future such issues? It seems like programming that acknowledges records complete or completed years ago or before 2017 could avoid a new change, reopening concerns with "ancient records."

STACEY: I can try to tackle that one. So, it isn't a product of Beta. It was a validation messaging problem with the classic system where appropriate information had not been included for age limits and so the system not having the proper validation and then having that validation added has created some issues with older records. And so I understand it's frustrating. You see something that you think is complete and then you find out that it didn't have complete information. The system validations are there to make sure - they're kind of a safeguard to make sure that you've entered what you're required to enter. But this information was noted as required previously and just wasn't. I think people were relying upon system validation to make sure that that information had been included. So what we would ask, is if you happen to notice this in a record, if you can go in, if you have a way to fix it, you know, please do fix it. We understand that there are circumstances that would prevent being able to fix it. And I would say write in to us for some advice. We may have some suggestions for how to work around those validation messages for you to update that information.

LOUISE: Thanks Stacey and I'm acknowledging-- Oh go ahead Nachiket.

NACHIKET: I would just add to Stacey's, you know, explanation she gave. I would just, about PRS Beta, I would really, you know, request all our users to, you know, when they are playing around with Beta, you know, and they notice that, you know, there is the data that they are-- or you know- they have reason to believe that there is an inconsistency between what they are seeing between classic versus beta. We really want to know about that, because again as Stacey mentioned in  early in our presentation, we can, you know, try so much and we do vigorous testing on -multiple rounds of testing- to ensure that, you know, there are no issues but that, you know, when it goes to the real world, you are the users and, you know, we really also rely on you to let us know about these issues. And we will, you know, if once notified we'll definitely act on them and try to resolve them ASAP. But yeah, that possibility is always there, and we can just try our best to, you know, make it at least disruptive for you.

LOUISE: Yeah. Thanks Nachiket. And yeah, I'm just acknowledging that Yang also had the same issues as Diane. So I think that they're happy to hear that you were looking into it. So next question is from Susan. Can we run Planning Report on the new Beta website?

STACEY: So I can answer this one too. I did demonstrate it on a slide that one of the options in the Saved View menu is the Planning Report set of columns. So you can filter for those columns. You can also filter for the Public Website View columns. So that would be the way to try to obtain those columns for a report.

LOUISE: Okay. Thanks Stacey. From another question from Msmolloy, Will the modern website allow us to delete records that have been made public? We have a situation where the current records should be registered under a different institution and at this time, we cannot delete the record as it has already been made public. And maybe Maureen, you might be able to answer this one.

MAUREEN: Yeah. I think your best course of action is to contact register@clinicaltrials.gov. Records can be transitioned or transferred from one sponsor to another sponsor within the system, Stacey or Nachiket, if you have anything to add, please do.

STACEY: No, I think you got it, Maureen. Thanks.

LOUISE: Thanks, Maureen. And we are coming up close to the end. So I'm going to try to squeeze in one more question from Lorna. We've gone in a circle with updating the record on a study and we keep receiving error messages/clarification requests. Yet we're not sure how to answer. Is there live help or guides that our website can use?

STACEY: So I think the best course of action in this case, if you're running into a situation where you really aren't sure how to address validations that you're seeing, is to write into the register@clinicaltrials.gov email address and hopefully we can help you that way. We don't have any sort of, you know, 1-800 number where you can get immediate assistance. But our information specialist team is really great about getting back to people very quickly.

LOUISE: Thank you, Stacey and that does help me transition to the close of our Office Hours. So we did receive a few other questions, but I'm going to encourage msmolloy and Jessica to please send in those questions to the email that Stacey had just mentioned.