Week Gone Wild

Here it is Friday and I am finally home after a fairly stressful day. This whole week has been quite stressful. I am working on a project with a company implementing SQL Server Reporting Services and man is this a difficult project at times. I have been working with reports for about 8 years now and it always amazes me how difficult they can become. When I used to work with Crystal Reports, it would sometimes drive me crazy and SQL Server Reporting Services is no different. Now, don't get me wrong, I am technically capable of designing and developing the reports, it is just the issues around end user specifications and data extraction. It is fairly common for me to be in situations where no one seems to know their data and I am tasked with figuring it all out while still performing my main tasks of report creation.

Now, I would like to clarify one fact here. I am by no means a DBA! I think I have stressed that fact in several projects and it never fails, I am thrown the awesome task of navigating my way through the database and creating complex queries that would drive an ape mad. I guess the worst part is that I love challenges too much and I haven't quite realized just how deep I am getting until after I am drowning. I keep telling myself that I can't let this keep going on, but I loose focus and fall right back into the pit and struggle each time to get out.

I am a programmer, think like a programmer and breathe programming! That is what I am. A .NET Programmer with a love for creating applications that provide solutions to problems that exist in the Enterprise. I like finding ways to code an application that can do what no other existing application can do. But, I find myself stuck administering band aides to wounds that need surgery. I guess the reason I enjoy reports is because of my creative nature, but there's a whole lot more to reports than creativity and those are the things that I guess I struggle with.

I chose Microsoft Technologies because Microsoft is very good at one thing and a lot of people don't like to give them credit for it. They create jobs! Microsoft's main purpose is to provide the tools for the populous which allows the populous to provide the solutions to their customer base. Prime example, I had an issue today that has just added to my already extensive list of future projects. I work in an environment that is tight on security and the systems I have to operate on use both SQL and Oracle. SQL Server Reporting Services is the tool of choice to extract data from these different systems.

Oracle is a complex data warehouse with about 15 different data stores housing terabytes of data. This makes for a very interesting day when attempting to create reports. I am tasked with creating queries that encompass about 5 to 10 table joins from potentially multiple data sources. To put the icing on the cake, I may at times have to extract data from a SQL Server data warehouse that is also several terabytes in size. This is where the pressure builds because these queries can take anywhere from 5 minutes to several hours depending on the tables I am joining to. Plus for some strange reason, security doesn't allow for oracle to use database links to SQL where I could run more efficient queries. So, I attempt to go the other way around by using a database link from SQL to oracle and find that utilizing any complex queries with OPENQUERY fails and when working is slower than running the queries in the oracle database. My time basically doubles.

Now, I see an opportunity here. How can I leverage my knowledge of both systems to provide a better solution then what is currently being used. There just has to be a better way than this. I attempted to formulate some plans today to attack this issue with little success. I figured if I could create some sort of middle component to allow me to perform this same process, things would be beautiful. What better technology to use than .NET, so I pull up a new project and started coding away. I had a great idea and had spec'd it out and started on the design and development. Then, the Business Analyst (BA) for reports shows up wanting the reports I have been working on yesterday. He has for some strange reason, promised his client some results by noon and needed to provide them with feedback. I informed him that I was having difficulties with getting data migrated over to oracle and I don't have enough permissions in SQL to migrate data from oracle and I didn't really want to do either since that would create duplicate data. So as far as I am concerned, those reports wouldn't be ready today. He of course didn't like that response and pressured me to provide some type of results today.

So, I had to put my ideas aside and return to the archaic process I have been using. That is pretty much the way things work everywhere I do contracts. Fixing problems for the long term aren't important. Preference is placed on temporary fixes. That of course saves them money short term, they think but they end up spending more money in the long term, because they will be calling me back next month to repair the same problem that I repaired last month.

Now, some tables have millions of rows that will potentially only supply me with a couple of records. So, my day is spent running queries and waiting for the results just to find out, there's no data. I make a change to the query and rerun to wait another 5 minutes to several hours. That isn't a problem for most people who don't mind milking the clock and getting their paycheck and going home. It just happens to be a problem for me since I want to get some work done sometime today.

Now, when I express this frustration to management and other associates, I seem to get the same response. This is the way stuff works around here and that just kills me. What can I do besides roll with the punches? So, I just develop an attitude of who cares, just come in and do your job and go home. Get your paycheck and be happy. Do enough to keep your contract and keep your client happy. Well, I must admit, today rubbed me the wrong way.

It is imperative that I work for myself. Every day I work on this project for this client, makes me more eager to finish developing my own software and do what I do best, provide software solutions.

walkins5