How to compute score and predict for outcome after N days

This could be seen as a "simple" binary classification problem. I mean the type of problem is "simple", the task itself certainly isn't... And I'm not even going to mention the serious ethical issues about its potential applications!

First, obviously you need to have an entry in your data for a patient's death. It's not totally clear to me if you have this information? It's important that whenever a patient has died this is reported in the data, otherwise you cannot distinguish the two classes.

So the design could be like this:

• An instance represents a single patient history at time $t$, and it is labelled as either alive or dead at $t+N$ days.
• This requires refactoring the data. Assuming data spans a period from 0 to $T$, you can take multiple points in time $t$ with $t<T-N$ (for instance every month from 0 to $T-N$). Note that in theory I think that different times $t$ for the same patient can be used in the data, as long as all the instances consistently represent the same duration and their features and labels are calculated accordingly.
• Designing the features is certainly the tricky part: of course the features must have values for all the instances, so you cannot rely on specific tests which were done only on some of the patients (well you can, but there is a bias for these features).
  • To be honest I doubt this part can be done reliably: either the features are made of standard homogeneous indicators, but then these indicators are probably poor predictors of death in general; or they contain specialized diagnosis tests for some patients but then they are not homogeneous across patients, so the model is going to be biased and likely to overfit.

Ideally I would recommend splitting between training and test data before even preparing the data in this way, typically by picking a period of time for training data and another for test data.

Once the data is prepared, in theory any binary classification method can be applied. Of course a probabilistic classifier can be used to predict a probability, but this can be misleading so be very careful: the probability itself is a prediction, it cannot be interpreted as the true chances of the patient to die or not. For example Naive Bayes is known to empirically always give extreme probabilities, i.e. close to 0 or close to 1, and quite often it's completely wrong in its prediction. This means that in general the predicted probability is only a guess, it cannot be used to represent confidence.

---

[edit: example]

Let's say we have:

• data for years 2000 to 2005
• N=1, i.e. we look at whether a patient dies in the next year.
• a single indicator, for instance say cholesterol level. Of course in reality you would have many other features.
• for every time $t$ in the features we represent the "test value" for the past 2 years to the current year $t$. This means that we can iterate $t$ from 2002 (2000+2) to 2004 (2005-N)

Let's imagine the following data (to simplify I assume the time unit is year):

patientId birthYear year     indicator 
1         1987      2000     26
1         1987      2001     34
1         1987      2002     18
1         1987      2003     43
1         1987      2004     31
1         1987      2005     36
2         1953      2000     47
2         1953      2001     67
2         1953      2002     56
2         1953      2003     69
2         1953      2004     -    DEATH
3         1969      2000     37
3         1969      2001     31
3         1969      2002     25
3         1969      2003     27
3         1969      2004     15
3         1969      2005     -    DEATH
4         1936      2000     41
4         1936      2001     39
4         1936      2002     43
4         1936      2003     43
4         1936      2004     40
4         1936      2005     38

That would be transformed into this:

patientId yearT age indicatorT-2 indicatorT-1 indicatorT-0   label
1         2002  15  26           34           18             0
1         2003  16  34           18           43             0
1         2004  17  18           43           31             0
2         2002  49  47           67           56             0
2         2003  50  67           56           69             1
3         2002  33  37           31           25             0
3         2003  34  31           25           27             0
3         2004  35  25           27           15             1
4         2002  66  41           39           43             0
4         2003  67  39           43           43             0
4         2004  68  43           43           40             0

Note that I wrote the first two columns only to show how the data is calculated, these two are not part of the features.