Olga’s Journal
September 8
I think that Tam’s idea to have a blog to keep our project’s journal and communication is very good. So here I will post my journal entries.
Tam and I have met a couple of times so far. Then we met once with Dr. Paul. Mostly we were trying to decide what exactly we plan to do and share our thoughts with Dr.Paul. Tam has posted our Statement of Work.
We will have a separate website for the project, a blog for discussion and actual website for our application. The website for the project will have information of the project’s progress, diagrams and so on.
We have registered a domain for our application. It is called Diatrack and domain at www.diatrack.com. It is nothing there yet, but we plan to work on the design of the home page.
I plan to use CSS styling to create both websites, so I am learning about it from the Charles Wyke-Smith’s book “A Designer’s Guide: Stylin’ with CSS”. I hope to apply this knowledge pretty soon.
Also one of our organizational problems is to decide if we will use CVS or Subversion. We know that we can have CVS from our school, but want to investigate Subversion.
September 10-16
We met with the Dr.Paul and mostly talked about GUI design issues. He recomended some books to read and point out that to create some kind of prototype is very important. I remember reading a very interesting book The Desing of Everyday Things by Donald Norman. In it the necessary principles of good design are explained. I remember he said that an object should provide a feedback and clear understanding of its own functions and state or design is broken. So I would like that our GUI design was follow this principles.
Also, I was continue to learn CSS formating and started to look for book about ASP.NET.
September 17-23
I started to learn ASP.NET by reading and doing examples from the book ASP.NET 2.0 Server Conrols and Component Development. So far I went through the chapters 1 and 2. Also I was thinking about what and how our GUI should display. On our meeting with Tam we talked more about the GUI design and how our product might be used. We thought that we will need to have a different roles for different users - some might need to have access only to the search page while other should be able to enter or modify the data.
September 24-30
On our meeting with Dr.Paul we discussed more of the web sites organization issues. So far we only register our domain and now we need to organise it. We decided to have a main address for our product and have a subdomain for the project issues. Also we agreed on our naming convection.
I almost finished the book about CSS styling and looking forward to create our websites. I have some ideas about how I want them to look and I want to share them with Tam.
October 1-7
I created couple pages for our project website. I am happy about how they look. I liked my exeperience with CSS - it seems to be work well, allows to create constant look throughout the pages and easy to add an accent if I want to. I have some problems when I was trying to post web-pages - I uploaded them, but they were not displayed when I was going to the web-sited. I found out that the issue was that the start page should be called index or home page. I fixed it and now it works.
I and Tam worked on the product website during our meeting. We did pair programming - it was good experience for me. It was nice to think and decide on something together and implement it right away. We took turns at the keyboard and hepled and corrected each other. It was nice that someone corrected all my typos. Sometimes, when I do something I could not easily see what is wrong, but the other person look and see it instanly. Also it was nice to share our knowledge.
By the end, we created and uploaded the product prototype. It just a prototype so later we might change it a lot, but the main structure probably will remain.
October 8-14
Dr.Paul remind us that we need to decide on code sharing. So I plan to look more closely into it. CVS seems to be nice and I am using it for the senior experience class throuht the college. I was considering the possibility to set up the version control through the college, but the college server was down so many times (this problem is fixed now) and I wanted to create everything myself. I search the internet and found that there are many free servers that we can use. Since they offer the same packages for CVS and Subversion I decided to set up Subversion. We do not plan to use it as command line, so I set up Tortoise to access it. I committed everything we have and I eager to share it with Tam.
October 15-21
While working on website with Tam I accidently deleted whole directory. But we were able to recover everything because we already upload it to the website. We should learn how to use Subversion more efficenly and commit more often to prevent such a situation.
I need to re-organize my project folders - since I had many spike solutions I already used the names I wanted to use for the real thing like diatrack (it was the reason why I deleted the directory - I thought it something else).
I was keep working through the ASP.NET book and found that I did not like this book - it had so many errors, it hard to know what needed to be implemented you cannot skip chapter and etc. So I order another book that seems to be good and looking forward to work through it.
October 22-28
I had a feeling that we kind of staying on the same place and not really sure what to do next. What is Dr.Paul is really good at is to help us to define our next small task toward to the big goal and he is doing it in a way that I think it is we are so smart that come up with this idea. So he just start talk about how we plan to store a patients data, what kind of data we plan to use etc… and our next step is to work on the actual data model.
The new book on ASP.NET is much better - the example actully works and it is not take a lot of effort to find out what I need to type in.
October 29 - November 4
Tam and I met to work on the data model. We both thought about it after meeting wiht Dr.Paul and as it appears have a different views. We easily agreed on what kind of data we want to include but we argued a lot about how it should be organized. I wanted to base everything on the patient’s visits, Tam - on the items like tests and have data fields for each item. Finally, I agreed with Tam (we cannot arguing forewer). I remembered from the Dr.Paul meeting that the user do not see the internal stucture of the database. We can design database one way, but when the user enter a data he/she can enter it as a visits or any other way depending on how the GUI is designed. So we draw the database schema and posted it on the website. Of course it can (and will) be modified later.
November 5-11
We met with the Dr.Paul and showed him the databse schema. He said it looks good and that our next step is to create object model. If we plan to do some of AI in order to find patients similar to the given one or to the given criteria, we want to load our data from the database in some form of objects. Then we can easy do any comparisons and manipulations of the objects. If we have an objects we can try simple query and AI and compare which one returns better results and more efficient. This lead us to another thing - the actual data - how to simulate it and so on. The data should be like real one, we cannot just put a bunch of numbers, but we will focus on it later.
November 12-18
With Tam we worked on the creating the object model. We both used to do our design while coding so it was kind of unusual. Anyway, we created an UML diagram of classes that will represent out object. I thought it was productive and we clarified a lot of details.
November 19-25
I did not do anything during this week, since it was out Fall break and I left for Thanksgiving.
November 26 - December 2 We met with Dr.Paul and were supposed to discuss our data model, but it was not finalize yet. So we spent our time talking about simulating the data, what kind of technique we can use, what kind of distribution data might have and so on. On of the difficulties that I see is that we can generate a data, but after that we need to create a different groups of patients out of it in order to test the similarity. The patients within the group should be different, but similar at some notion, we cannot create a group of similar patients by duplicating one over and over. Also the patients can be similar or different in the respect of time - if some one wants to compare the similarity on the time base.
December 3-9
With Tam we started to collect some data in order to simulate it later. We use our database schema and looked what kind of tests and data can be measured. Then we looked for the actual numbers for those tests like what is the normal values, how low and how high it can go. I thought this task will go much faster, because I have some knowledge about the data, but as we found out the unit conversion is kind of issue and consume plenty of time. We did less that we wanted and plan to return to it later.Anyway, it close to the end of the semester and we both have a finals.
December 10-16
We met with Tam to make sure that everything we have is up to date. I felt like we did not accomplish much - we mostly were deciding on everything. But when we were talking we recollected the thing we did and it sounded not so bad. Anyway, I am looking forward to the Winter break during which I plan to do some work for the project.
December 17 – January 14
During the winter break I was working on data generation. I spent a lot of time (at least 8 hours a day for 3 weeks) doing it and still not completely done with. The problem is that the data should be realistic enough that it did not distract from functionality during the demonstration of the system. And also a large amount of data is needed for testing web pages, queries and visualization. The test data is generated by a set of rules and probabilities. I had to do a lot of research and use may experience as a doctor in order to create this set of rules. To see additional information about data generation see Test and Demo Data in the Specification Document.
Also during the winter break the Specification document was written. At first we did not plan to have documentation, but as time progresses, we start to have problem what we decided to do. Our project is big and we were working on it more than one day, so we could not keep the every detail in our minds. Of course we have some kind of documentation before, but it was not organized, on different pieces of paper in different places. Now, we have a well organized document that we will try to keep updated throughout the project.During the winter break I was working on data generation. I spent a lot of time (at least 8 hours a day for 3 weeks) doing it and still not completely done with. The problem is that the data should be realistic enough that it did not distract from functionality during the demonstration of the system. And also a large amount of data is needed for testing web pages, queries and visualization. The test data is generated by a set of rules and probabilities. I had to do a lot of research and use may experience as a doctor in order to create this set of rules. To see additional information about data generation see Test and Demo Data in the Specification Document.Also during the winter break the Specification document was written. At first we did not plan to have documentation, but as time progresses, we start to have problem what we decided to do. Our project is big and we were working on it more than one day, so we could not keep the every detail in our minds. Of course we have some kind of documentation before, but it was not organized, on different pieces of paper in different places. Now, we have a well organized document that we will try to keep updated throughout the project.
January 15 – January 21
On Wednesday of this week we had a meeting with Dr.Paul. We were discussing the problems with data generation and were talking about where we are in the project. We decided to devote two more weeks for data generation and then move to the next part. Also, Dr.Paul wants us to give him walk through of some part of code of data generation.
January 22 – January 28
We work with Tam on picking up the part of data generation code for walk through. The data generation code is very complex and had at least 30 classes. So we wanted to pick something representative and at the same time not very big, so Dr. Paul could understand the general idea and method used. We finally picked a piece that we both like. I hope next week we will impress our advisor.
January 29 – February 4
We had our code walk through. I have never done it before. I wrote a lot code for homework assignments but I never had to actually explain why and what every single line of code mean! It was extremely useful for me. I learn a lot. We did not discuss anything else, because we got exhausted. And we actually find one small bug.
February 5 – February 11
I was keep working on data generation. There is still a lot of work and improvement could be done. But I think I need to finalize it set the priority, left something unimplemented and populate the database. For example in the real life some patients are dying, some percent of them men, and some women. They died from the different reasons – accident, diabetes, heart disease, cancer and so on. On the other hand new patients are acquired by the doctors. In addition to it some patients are moving from one doctor to another and so on. So I decided do not implement this part and have out data kind “freeze” in time.
February 12 – February 18
We had our meeting with Dr.Paul. We were discussing our plans for the rest of semester. I think I unintentionally confused them by saying that the test data is in the database, it is not yet. But I hope to populate the database tonight. It is big deal for me. And I did not put data in the database before because I wanted to test it and see that generated data is the data I want (I was using NUnit as much as I could, but there is a lot rules and randomness so it hard to test). So instead of populating database I was populating dataset and then query it to see what I have got. This solves the problem of deleting the data from the database every time something went wrong. But I think I am ready to actually put the data in the database. Anyway, our plan is work on search page and case based reasoning until spring break, than devote some time to visualization and then work on final paper. It is not much time left, so we decided to work in parallel and my part is search page and queries behind it.
February 19 - February 25
Finally I was able to populate database with the data. I thought it should be easier like one line of code dataset.Update(), but it was not. Anyway it is done and I move to the search page.
February 26 – March 4
We meet with Dr.Paul. Dr.Paul and Tam agreed with my decision to change the interface of search page. I wanted to simplify it a lot. Our previous design included a lot of “cool” controls. For instance sliders that allows to set search parameters. But all those controls are user controls and have to be implemented by hand that would take a lot of time. So for now we will have simplified version where user can enter the parameters and check and uncheck some buttons. Also we discuss with Tam what results she can have from the search query for further processing.
March 5 – March 11
We meet with Tam, I showed the implementation of the search page, and she liked it. Also we worked on what information she can use from the search query for aggregation and generalization of the data.
March 12 – March 18
I started to work on search query. One of the issues with it is that the query should return results throughout a time. For example, if user asked to return all patients between ages 30 – 35 with bmi of 35, than the query should return not the patients who is 30-35 now with such bmi, but all patients who ever had bmi 35 and at those times were 30-35 ages old. I started with part of query that will return patients based on age and simple search criteria as duration of diabetes and bmi. I had to denormalize the database because in order to access some metric values I had to know the VisitID. So I add VisitID, dates and PatientID to some tables.
March 19 – March 25
It is spring break, but I have a lot to do. Since I denormalized the database, I had to change a lot of underlying code: data generation, insert, update and delete queries and store procedures - it is a lot of work. So, it will keep me busy.
March 26 – April 1
We met with Dr.Paul. We discussed the progress on search query. I explained the reason of database denormalization. I keep working with the search query. Also Dr.Paul told us about the regional conference where we can present out work. I am exited about it. So we need to write and submit an abstract by April 15. We discussed the topics we might want to include in the paper. Dr. Paul said he would write first draft of an abstract.
April 2 – April 8
I am almost done with the search query. It looks very complicated. The problem is that it runs for 50 seconds. It is not acceptable. And database has only 20 patients with about 6000 visits. No user will want to wait long time to see results. So I need to optimized query somehow. I read that indices will improve performance. I decided to try it out. It works well. Now I have the query’s results in no time.
April 9 – April 15
It is not much time left to the end of the semester. Dr. Paul sent out the draft of the abstract. I added some minor changes and we submitted it. I am sure that they will accept it.
April 16 – April 22
We had out meeting with Dr. Paul. The good news is out abstract is accepted. So we need to write a paper. It is so much that we want to say about this project. So much we learnt and discovered and so many ideas we have and want to share. The paper size is 15 pages double spaced. It seems to be not enough for the amount of information we have. So we will need to focused on some and skip other parts.
April 23 – May 5
I was not doing much. It is the end of the semester, I have finals and I am graduating. But I am thinking about the paper and final report for the CREU.
May 6 - present
We received the request for final report. So I was checking if out website is up to date, and was writing the final report. I think we did well. We accomplished a lot and I am glad that I was participating in this research project. I like our project and plan to enhance it in the future.