Informatics and Data Science Projects

NLP API for extraction of structured data from freetext palliative care consult questions

I developed a rule-based natural language processing tool using MedSpacy which can extract the diagnosis and reason for consult from free-text palliative care consult questions.

This tool is planned for use as part of our department’s push to integrate with the Palliative Care Quality Collaborative, a nation wide database of palliative care programs used for benchmarking and QA/QI initiatives.

The code can be used within Jupyter Notebooks, but for ease of deployment and transportability I also turned it into a dockerized FastAPI and deployed the API to a DigitalOcean Droplet.

View code on GitHub
Try API in Google Colab
View API Docs

XGBoost and Transformer Time Series Models to Predict Emergency Department Visits

Using publicly available data, I trained an XGBoost model to predict daily emergency department visits at UC Davis.

I engineered time series features (lags, rolling averages) to maximize the xgb model results.

This is an ongoing project, with plans to include more robust external data, such as holidays, local major events, air quality, and internet search data.

I have also begun working on a transformer model for the same task.

XGB model results

GitHub Repo of Model Development Notebook
PDF of EDA Notebook
GitHub Repo of Transformer Model

Sentiment Analysis of Student Doctor Network Specialty Forums from Inception through 2023

To see how sentiment of various specialties has changed over time, I scraped forum posts from the Student Doctor Network subspecialty boards using BeautifulSoup. I then analyzed the sentiment of each post using a HuggingFace Transformer.

An abstract using this data was accepted as a quick shot presentation to the American College of Surgeons Clinical Congress 2024.

Here are the results for each specialty, averaged by year, rendered using D3Blocks (click the specialties to see their plots):

Jupyter Notebook of Scraping/Sentiment Analysis Script

(SDN has since changed to dynamically loaded content, so this scraping script no longer works)

Department QA/QI Dashboard

I created a Streamlit dashboard to explore departmental consultation data for our inpatient palliative care service.

The ETL process is automated, so newly generated EHR reports can easily be added to the existing data.

The NLP program above is utilized to enrich the data to allow for more granular analysis of patient diseases and reasons for consults.

The data is filterable by all columns and all visualizations automatically update to reflect filtering.

Examples of the visualizations within the dashboard, values have been edited out

Page template forked from evanca