Another series, initiated in a Linkedin post.
The PMS process is nothing more than a research project focused on answering specific questions related to the product, with the objective of defining actions based on the research results. To do so, it must be planned and designed as a research project, with fundamental questions that the company wants to answer, use data research methods (most of them from the social sciences field), analyze the data and create a report of (the PMS report) with the analysis and conclusion on which predefined actions (from the planning) shall be carried out.
Some of the literature I use as a basis for explaining the process:
– An Introduction to Qualitative Research – A great introduction to research concepts and methods
– Research Methods in Social Relations – A basic guide to applying research in the social sciences
– Doing Business Research and Management: An Essential Guide to Planning Your Project – An introductory guide to the application of research in business
For sampling, I also like this book: Sampling Methodologies with Applications (Chapman & Hall/CRC Texts in Statistical Science).
– Researching Business and Management: The Roadmap For Success – A more detailed guide to the application of research in business
Finally for this first part, here is a list of activities required to implement a post market surveillance (PMS) system for medical devices (a better formate file can be downloaded here: https://www.medicaldevice.expert/wp-content/uploads/2019/10/Example-of-MD-PMS-Activities.pdf.
Post-Marketing Surveillance – Alternative definition – Systematic business research project to collect and review data from the experience gained from medical devices placed on the market and to define actions by analyzing this experience data
The rigor of the process(which offers a general increase in the quality of results) needs to be present to ensure, among other things, that no bias is introduced, however, the process as a whole need not be so rigorous as a systematic review/clinical evaluation – it needs to be at a minimum as rigorous as a generic systematic research project. However, In the case of clinical data (PMCF), this part of the process needs to follow the same rigor as the systematic review / initial clinical evaluation, as it is nothing more than an update of the systematic review / initial clinical evaluation.
Systematic Process – General (adapted from Researching Business and Management, Harvey Maylor and Kate Blackmon)
D1 – Defining your business research project – Develop PMS Plan
D2 – Designing your business research project – Develop PMS ProtocolD3 – Doing your business research project – Perform PMS ActivitiesD4 – Describing your business research project – Develop PMS Report
D5 – Acting based on your business research project – Perform activities and feed back to PMS
|Project definition||Research topics, research problem||PMS Plan|
|Project design||Research questions, Methods for gathering and analyzing evidence and testing knowledge claims||PMS Protocol|
|Project execution||Knowledge claims, evidence, analysis and interpretation||PMS activities|
|Project description||New knowledge||PMS Report|
|Actions||Action upon knowledge||Diverse, report back to PMS, update PMS report if required|
P – Preliminary planning
P.1 – Answer some general questions
– Why is the company doing this research project, and what do the company want to get out of it?
– What is the research problem?
– What do the project want to find out?
– Where is the information that project want to find out?
– Who will want to know what the project find out?
– How will the project be assessed?
– Who will be carrying out this research project?
– When can the project start, and when must the project be finished?
– How should the project work with any external stakeholders for my work?
– What are the requirements of any assessment body, such as the NB or RA?
P.2 – Draw up an activity list, including dependencies
Generic example focused in surveys
Investigate literature and write literature review
Pilot test survey
Enter data from survey into spreadsheet
Analyze survey data
Write up results of data analysis
Produce first draft of product report
Seek comments for draft
Revise report into final draft
Perform actions based on report findings and related indicated actions
Feedback to PMS
Update report (if required)
P.3 – Draw a timetable (for example, a Gantt chart)
P.4 – Draw a network diagram for the activities
D1 – Defining your business research project – Develop PMS Plan
D1.1 – Set up project goals and outline plan
D1.2 – Define research problem
D1.3 – Create a framework for classification of research topics
Example topics adapted from NB-MED/2.12/Rec1
– Performance (including clinical performance
Feedback about patient clinical benefits
Feedback about indications for use
Knowledge of new performance trends evaluation
Knowledge of long-term performance/reliability and/or chronic complications
Knowledge of performance in different population of users
Feedback about use with other devices, accessories and substances
Confirmation (or not) of risk analysis
Knowledge of ways in which the device is use incorrectly
Information of incidents related to the device
Identification of vigilance reports
Information of trends
Improvement of product quality
Detection of manufacturing problems
Feedback about Instructions for Use
Feedback about necessary training for users
D1.4 – Prepare PMS plan and submit for approval
D2 – Designing your business research project – Develop PMS Protocol
For each research topic (see (D.1.3)
D2.1 – Identify background information related to the specific research topic
D.2.2 – Identify the need for information
D.2.3 – Identify the general goal of the information use
To update the benefit determination / risk and risk management, design and manufacturing information, instructions for use and labeling;
To update the clinical evaluation;
For the identification of preventive, corrective or field action needs ;
To identify the opportunities to improve usability, performance, and product safety; Where appropriate, contribute to the post market surveillance of other devices
(May also tie with specific regulatory requirements)
D.2.4 – For each research topic, create well-constructed research questions
Initial literature and medical device design information review
Use a mind map to refine your research topic into questions
Draw a hierarchy of concepts to define variables
Use Venn diagrams to refine your research topic into question
Review and validate research questions
Are the research questions giving significant contribution?
Are the research questions biased?
Are the research questions self-answering?
Are the research questions unanswerable?
D.2.5 – Identify research approach
For each research question, identify if the research approach will be scientific or
|Characteristic||Scientific approach||Ethnographic approach|
|Questions that can be answered||
What, how much
|Associated methods||Survey Experiment Databases||
Direct observation Interviews|
|Data type||Predominantly numbers||
D.2.6 – For each research question, identify who can provide the data – the sources. Possible sources – doctors, patients, medical staff, distributor, logistics operator, training staff, sales staff, experts, literature, internet, regulatory sources
D.2.7 – For each research question, identify how the information will be used in practice (for which process / activity / team / document / etc, what time and what is the expected action of the information)
D.2.8 – For each research question, identify whether the research will be quantitative or qualitative
|Quantitative versus qualitative research design|
||Quantitative approach||Qualitative approach|
|Number of Observations||Many||Few or one|
|Research Question||Who, what, where, when||How, Why|
|Variables||Specified earlier, based on theoretical concepts||Emerges from the study, based on grounded research|
|Data collect||One variable at a time||One case at a time|
|Analyis||Level of variables and relationships between them; statistical analysis||Event or process pattern discovery|
|Objective||Generalizable for observations or contexts beyond the sample||Generalizable for theoretical concepts|
For quantitative research
Identify if someone has collected the information. If yes, perform secondary analysis (eg as part of document analysis)
If not, assess whether it is necessary to show cause-and-effect relationships.
If not, use survey
If yes, use experiment
For qualitative research
Identify the type of information
If it is necessary to understand what has happened over time and from that to better understand change without direct involvement in data collection, use indirect data collection (eg as part of document analysis)
If it is necessary to see what is happening now and better understand what is happening, with data collection but no interaction with the study subjects, use observation
If it is necessary to explore problems as an external part of the process and unpack the research question through discussions, use interview or focus group
If it is necessary to understand and feel what is happening through personal experience, use some participatory method
D.2.9 – For each research question, identify methods by which information will be collected based on previous decision
– Experiment (eg PMCF studies, internal testing of new products (design), failure analysis of new products (design))
– Observation (eg user reactions during training programs, contextual inquiry)
– Individual Interviews
– Focus Groups
– Document analysis (eg literature review, different user feedback from complaints, either directly with the manufacturer or through sales personnel, registration implants / tracking products; experience of similar products manufactured by the same or different manufacturer, customer complaints and warranty claims, regulatory feedback)
– Case study (internal testing of returned products, analysis of returned product failures, explant recovery studies or exchanges) (case study is not exactly a research method but an application of several methods to specific cases – it focuses on which data to collect rather than how)
D.2.10 – For each related research question and method, develop the method (eg, create survey, plan experiment or observation, define literature review)
D.2.11 – For each research question and related methods, define a systematic data recording model
– Citation – Full study details or information
– The purpose of the study or information – Why did the authors do it and why did they think it was important?
– Theoretical framework – How does study or information fit the literature
– Sample – Who did the authors study or information?
– Research scenario – Where was the study or information conducted?
– Method – How was the data collected?
– Data – What data was collected?
– Analysis – How were the data analyzed?
– Results – What were the main results of the analysis?
– Conclusions – What did the authors say about the results?
– Limitations – How should we interpret the study or information?
– Significance – What did you learn from this study or information?
D.2.12 – Organizing access to data sources (see D.2.6)
Use contacts to gain access
Identify and address ethical issues for field research
Identify and address ethical issues related to data reporting
D3 – Developing Your Business Research Project – Performing PMS Activities
Use systematic approach to quantitative data
Decide which variables and characteristics of respondents will be captured as data
Decide how it will be measured for each variable or characteristic
Decide how to record each measurement
Structured Approach – Deciding which data to collect as it will be analyzed> Collecting data> Structuring data> Logging and clearing data> Analyzing data> Interpreting analysis> Reaching conclusions> Reporting results and related actions
Use descriptive statistics to summarize and represent raw data
For more than one variable, use simple hypothesis testing and bivariate statistics, such as correlation coefficients or simple regression analysis.
Use advanced techniques such as multivariate analysis if necessary
Ensure data is traceable, reliable and complete
Define criteria traceability, reliability and completeness criteria for data
Validate which data meets criteria
Analyze qualitative data
Define which approach to use for qualitative data – unstructured or structured
Use models for qualitative data analysis
For example, Kolb learning cycle
Get Data> Get Familiar with Data> Spend Time Considering the Questions Raised> Reorder and Summarize Data> Extract Key Concepts from Data> Check for Concepts Occurrence or Recurrence> Observe Pattern Emergence> Analyze Patterns Fit Data
Analyze if data fits the structure
Use models for qualitative data analysis
For example, concept extraction, mapping, logical diagrams
Evaluate the quality of findings
Does the way you present your findings give the impression that they are well founded?
Does the work reflect the reality of the problem or situation being investigated?
What is the applicability of the findings to the wider world, beyond the one you considered?
Would it be possible to repeat the work and get the same or similar results?
D4 – Describing Your Business Research Project – Developing the SMP Report
Interpretation of quantitative data
Use techniques like tables, graphs and others
Interpret the analysis of statistical results
Describe applied statistical test
Interpret empirical data
Credibility, validity, generality and reliability
Interpretation of qualitative data
Organize data categorically or thematically
As a narrative
As a thick description
As a personal journey
Link results to literature
Develop findings and action recommendations
Creating Summary Tables
Identify actions needed from findings (see D.2.7)
Write PMS Report
D5 – Acting on Your Business Research Project – Perform Activities and Feedback PMS
Feed back to PMS for completion
Review report if required