The nuts + bolts of cleaning data through the eyes of a Datanomy data analyst

As a data analyst working in a healthcare, cleaning data is an essential part of my job. Before I can start working with the data to gain insights and make recommendations, I need to make sure that the data is accurate, complete, and consistent. This involves a number of steps, which I will outline below.
​
First, I start by inspecting the data to get a sense of its structure and content. This includes looking at the data types of each column, checking for missing or null values, and identifying any inconsistencies or errors. For example, if a column contains patient ages but some of the values are negative or extremely high, I know there is likely an error in the data that needs to be corrected.

Next, I begin the cleaning process by addressing any errors or inconsistencies in the data. This might involve replacing incorrect values with corrected ones, or dropping rows or columns that contain unreliable data. I also make sure to handle missing or null values in a way that doesn't bias my analysis. This might involve dropping rows with missing values, filling in missing values with the mean or median of the column, or using a more advanced technique like imputation.

Once I have corrected errors and addressed missing or null values, I move on to formatting the data in a way that makes it easier to work with. This might involve changing the data types of certain columns, combining multiple columns into a single one, or creating new columns based on existing data. I also make sure to standardize the data, which involves ensuring that all values are consistently formatted and easy to compare. For example, I might convert all dates to a uniform format, or convert all measurements to a standard unit of measurement.

Finally, once the data is clean and properly formatted, I save it in a way that makes it easy to access and work with going forward. This might involve saving it to a database or a flat file like a CSV, or even creating a data model that can be used to query the data more easily.

In summary, cleaning data as a healthcare data analyst involves a number of steps, including inspecting the data, correcting errors and inconsistencies, formatting the data, and saving it in a usable form. By following these steps and paying close attention to the details, I can ensure that the data is ready for analysis and can be used to make informed decisions about the care and treatment of patients.