Policies and Procedures

As a CIO, I've been writing policies, guidelines, and procedures for a long time.  My technical background did not prepare me to write these kinds of documents.  I suppose my general education and business background did provide some preparation.  I worked in a law firm many years ago and had to write guidelines for staff members that satisfied the partners (who had a high writing standard).  I have a Master's in Public Administration and we covered many similar topics in that program.  Still, I've never really studied "writing information technology policy."  For a while I've been thinking we need a uniform policy and compliance guideline, but I'm not sure who could write it.


Several years ago, Rodney Petersen wrote a useful article for Educause Review:  

“A Framework for IT Policy Development” (http://www.educause.edu/ero/article/framework-it-policy-development). Another useful piece from the Educause is by M. Peter Adler: "A Unified Approach to Information Security Compliance"  http://www.educause.edu/ero/article/unified-approach-information-security-compliance.

This SANS chart is good, but you have to interpret it for policies:  http://www.sans.org/whatworks/applicable_sections.php.

This week I attended the Educause Campus IT Policy Workshop, led by Greg Jackson, Educause Vice President, Policy, Jarret Cummings, Educause Policy Specialist, and Kent Wada, Director, Strategic IT Policy and Chief Privacy Officer, UCLA.  This was truly a workshop, with strong presentations followed by case studies and discussion.  Links and materials to useful references were provided.  The event was a positive learning experience, even for someone who has been writing policy for a while.

As a result, I came back and reviewed our university policy template again.  Our university policies are posted here:  http://www.oakland.edu/policies.  Our information technology policies exist within the university framework, but I isolate them on our web site for presentation with other procedures and guidelines:  http://www.oakland.edu/uts/policies.    

I'm in the process of updating a half dozen policies to incorporate feedback from the external PCI auditor.  I am trying to organize my "policy thinking" and I am going to try to bring some consistency to the documentation. The PCI auditor had recommended that we separate all PCI elements into one PCI policy, but after the workshop, I am more convinced that the separate policies that are IT-centric still makes more sense, particularly for a university.  Too many components for regulatory compliance overlap.  It doesn't seem to make sense, for example, to have separate policies for FERPA, HIPAA, and PII data, when it all comes down to information and data security.  I also want to create policies that are not obstacles to what the university is trying to accomplish.

Once a policy is developed, our review and approval process is described here:  http://www.oakland.edu/uts/policies#governance  It is quite of bit of work to shepherd a new policy or a policy update through the process, but it is worthwhile. You hear different perspectives and sometimes you realize the policy needs to be worded differently.  Shepherding a policy also presents an opportunity to get support for the policy, particularly when the subject matter is difficult.

To help organize my thinking, I've created a checklist, posted here:  Policy Checklist

Anybody want to talk Policy?  

Changing Software Support Model

The changing role of the CIO and the central IT organization is certainly a hot topic at the moment.  One item I have spoken about is the changing software model.  I do believe we are moving away from the traditional IT requirements-based approach that worked (and probably still works) well for software development, to a model based on consumerization.  Our constituents fall in love and buy software.

A few years ago I participated with Educause in writing about reviewing software and managing alternative sourcing at Oakland University.  We marveled in that we looked at over 25 pieces of software in a year, and most were not coming through traditional project methods.  We had to review our purchasing process.  This past year the university community looked at over 325 pieces of software, that we know about.  We've done pretty well at making sure we are part of the purchasing process so that we avoid redundant purchases and software installations and support aren't unexpectedly dropped in our laps.  Some software applications are installed on campus, and some solutions are hosted.  Some are desktop installs, some are server installs, and now we have the App family.

Nevertheless, there is an emerging issue about software support.  Obviously, my staff of 33 IT professionals cannot be experts on supporting over 300 pieces of software.  The university cannot hire a system administrator or application support specialist for each piece of software.  Our traditional support expectations do not scale with that volume of purchases. We can't even be the sole-vendor contact. Yet I had an uncomfortable situation last week where a senior staff member told me that calling a vendor for support was "not a lean process."

In discussions with my staff, we've identified about a dozen key software systems that we've installed where we only offer hardware and operating system support.  There may be a disconnect between the services we offer and the expectations of our community.

I'm trying to put together tiers of support for software, something like:

• Outsourced.  Contacts go directly from client to vendor.  Central IT gets involved for contract review and assists with resolution in technical disputes.
• Campus install in datacenter with full support.
• Campus install in datacenter with some support.
• Campus install in datacenter with only operating system and hardware support.
• Departmental server install.
• Local device install.

As I'm looking at this, there is a developing support matrix.  In addition to location and local support resources, we also have to look at server, storage, operating system, web server applications, identity access management, database management applications, code management architectures, security applications, backup services, and the actual software application.  Server and storage architectures are a lot more complicated to manage with the virtualized environments we now install.

In some cases our central IT operation provides and supports the entire list.  The closest is our ERP implementation, even though we do push using the vendor support options (including mailing lists and online community) before contacting us.  In other cases, we split the work with another unit; for example, for Moodle, we provide the server, storage, operating system, identity access management connection, security applications, and backup services.  Our E-Learning operations does the rest.  We jointly work on tuning.

In other cases, particularly for research, we are hosting hardware and really not doing much else.

The issue we see emerging is when a software package starts locally, in a department, and gains adoption.  We then need some sort of escalation and approval process to move the software to a support model.  A few years ago we looked at this from a server support model, with levels described on our web site.  We really need to rethink this for tiers of software support, and the governance model that moves a software solution to a higher tier of support.  This needs to be logical and rational.  We need to include the staffing needed to provide support in the evaluation.  That piece is really missing, as we've been resourced constrained so long.  

We can't keep adding solutions to the VM architecture without analyzing the impact on staff resources, and we haven't fully identified the impact on staff.  We can respond to a new server request with agility in the VM environment.  When we do so, we need to be clear that it doesn't mean we can answer questions about the software internals or how the application really works.  As we get a more and more complex virtual environment with expanded storage and complex disaster recovery scenarios, we need systems engineers and storage architects, but these additional staff will still not be able to answer the questions about how the application really works.  Then we get a community disconnect;  we got additional technical staff, but the client support expectations may still not be met.  We aren't there yet, but we seem to see new challenges for communications, particularly communications needed to justify additional resources.

Our path is not yet clear, but we are thinking about it.

Who needs the web?

http://www.nytimes.com/2012/04/11/technology/instagram-deal-is-billion-dollar-move-toward-cellphone-from-pc.html?_r=1&nl=todaysheadlines&emc=edit_th_20120411 

The timing of this article connected to my thinking about future projects on our enterprise systems team and allocating resources from that team to the wide variety of identified projects.  I had lunch yesterday with the Director for Enterprise Systems, Lori Tirpak, and two team members who worked on mobility projects for the last year.

The team has finished a strong set of projects this past year. Projects included improved messaging to students about advising and progress to degree.  Also, the team upgraded the portal architecture to the uPortal platform that is mobile-ready.  Finally, the group completed and launched the first OU App in the Apple store.  We spent time talking about projects for the coming year, and we talked about when we might no longer support the portal and instead totally focus our resources (a team of 8 people) on mobile initiatives.  Obviously, something like a transcript isn't going to translate well to a smartphone.  But many services will transfer to mobile devices.

The phrases that caught my eye in this article:
“For decades, the center of computing has been the desktop, and software was modeled after the experience of using a typewriter,” said Georg Petschnigg, a former Microsoft employee who is one of the creators of Paper, a new sketchbook app for the iPad. “But technology is now more intimate and pervasive than that. We have it with us all the time, and we have to reimagine innovative new interfaces and experiences around that.”


"Who needs the Web?"


It reminds me of the era when we kept our old mainframe databases running, due to the cost of transitioning those databases to new platforms.  Instead, we dropped the mainframe screen interfaces and replaces the user interfaces with new designs and platforms.


So we can keep those huge ERP databases or learning management systems behind the scenes.
Then focus on small service bites that are important in the moment and that can be provided by mobile devices.  That has to be the priority.  Using the points in the article, put your resources into mobile from the start, rather than starting with the web and redesigning for mobile.
Any left over resources - and that isn't much on our team - updates the web experience.  That is quite a shift in thinking.  


This really has us thinking about project priority for the next year.

Academic Records and the Changing Education Model

I think, increasingly, what we will be buying from individual college and universities, is the credential, not the full learning experience that we associate with traditional college.  How we store records supporting the credential, and put those records together to indicate evidence of learning, may change in the not too distant future.

Imagine a space where you, as an individual, take some courses from MIT or Stanford, online, open and free.  You build your knowledge for knowledge sake.  There are two reasons we pursue knowledge:

• personal growth
• obtaining meaningful work that pays a salary that meets our lifestyle expectations

The open course-ware model works perfectly for personal growth.  But how does an employer know that you've assembled a body of knowledge that leads to you being a talented individual that is knowledgeable, capable, and hire-able?  Right now, degrees provide that university tattoo of knowledge.

Today, each university owns the academic records that make up that tattoo.  The student has to meet the requirements of that university to obtain the tattoo.

What if the world was different:
Each student owned their academic records, rather than the educational institution owning the records.
Students might hire a company that is an "academic records repository" or "academic records bank".
Some records in the repository are graded; some are not.
Some records may be combined to present a "Badge" of learning.  A Google search on the two words "learning" and "badge" show this growing concept.
Some records, graded and accredited, may be assembled into a traditional degree.
The student assembles a personal tattoo that is not defined by a university, but by the student.

What is the role of the university in this environment?  The university may offer some courses for credit and some as open (like Stanford is doing now).  The student may obtain the knowledge in either path.  One path, the credit path, achieves a higher standard of knowledge certification (i.e., grades).  So what the university is really providing is knowledge certification, not just a diploma that represents completion. 

What if I, as a student, can build knowledge from several sources, some certified, others open, and record that with the third-party academic records bank that is independent of any particular university?  The records bank doesn't certify, but provides the transcript of all the work, instead of the university transcript.  The university credit transcript would feed into the academic records bank, as well as open course work, technical certifications and whatever.  The employer would request knowledge verification from the academic records bank, not any one university.

The "Big Blue Button" idea of medical records could apply to student records.  But if institutions could send directly to the academic records bank, what would be the point of the "Big Blue Button"?  Perhaps to make a record withdraw to send to a potential employer.

We are now sending all academic records to the State of Michigan for storage in a state-wide database.  Perhaps certifications and other learning mechanisms could feed into that database.  So perhaps, down the road, the state will provide the academic records bank.

What does this do to the academic records we store on campus?  Is there any reason to keep years and years of campus history if we can store the records in a single centralized bank?

There is potential for a shift in where and how we store student academic history.

Storage

Steve Glowacki, Director Systems Engineering at OU, and I have had several interesting conversations since the holiday break.   Steve ended up doing trouble-shooting and problem resolution (with his team) over the break.    I thought I would interview Steve for this blog post.  This follows on to my earlier impacts post, as we talk about data and content, and the need to store ever-larger files.  Nothing is ever deleted, it seems.  What is back-up and disaster planning in this growing data environment?  We had a discussion about problems and tactics.

Steve, can you describe some of the challenges experienced over the break?
We noticed that the back-up amounts had basically quadrupled started just before the holiday.  We went from approximately 500GB differential per night to usually slightly more than 2TB per night.

Did you find out the source and stop it?
Finding the source wasn't terribly difficult but we couldn't stop it.  It was valid stuff to back-up.  There were a lot of changes going on.   Over the break, we allocated an additional 3 TB of storage to back-ups, which was everything that remained within the architecture, and added 10 additional LT04 tapes, which the system promptly consumed.  We placed an emergency order for 20 additional tapes while still on break.

What next steps did you take when you returned?
We started reviewing how to consolidate tape utilization for optimized tape utilization.  As tapes are written to, the tape is consumed or full.   Then as data ages off, portions of the tape are freed, but the physical tape is still consumed.  Out of 130+ tapes, a good portion have low percentage physical utilization.

What we are looking for now is a process or procedure that will consolidate the data in use, distributed across several tapes, to a single highly utilized tape.

The other thing we are looking at is migrating to a new complete new architecture for backups, restoration, disaster recovery and data de-duplication.

What are you looking at?
The architecture is based on VMWare / NetApp, so the backup environment needs to work very closely with that architecture.


So, a minute ago, you said you were trying to review something and said:

"In some obscure way you log into this thing and control backups, restorations, bare metal restores, or tape archiving....  The tech team is having discussions about how this all works."

Tell me more.

What I found funny was that it took two university engineers and a vendor sales engineer three days to find the compatibility list for tape libraries.  It is complicated.

What makes it complicated?

Tape is becoming very limited use.  

What should we be doing?

Remote site de-duplicated replication is one option.  The remote site may be here or the cloud.  Remote site is likely phase two.  Phase one is implementing solutions to work with the VMWare / NetApp architecture.  Phase two is the remote site capabilities.  The reason we are breaking it into two pieces is timing and procurement process.  De-duplication and how that is done technically is extremely important to consider.  

So several purchases have been made over the past couple days.

We purchased another shelf of high speed disks.  This will allow us to optimize server performance and through-put.  Looking at the total input/output per second across the two types of shelves we have, and made a decision to improve performance by going with smaller, faster disks for specific services.  For example, we have a virtual server which may have enhanced performance by being located on faster disks while the connected storage may be on slower disks.

A lot of analysis about what to put where...

It's an ongoing thing.  

And your second purchase?

Several software options driven by recent requests.  One request was video streaming, so one of the options is for turning on native shares for CIFS.   Another software request was for NFS, to allow for UNIX-based mounts.  This has the potential to additionally augment throughput.  Another is SnapVault, but it is just for swarming snapshots, and that takes us back to where we started this conversation.

Is your head spinning?

A bit.  It's my job, though.  A lot of conversations will be needed with the Network team too.  

Favorite News Stories - 2011

My favorite tech-related news stories of the year:

The year of nature web cams
This was the year that watching nature via web cam caught my attention.  My favorite story was of Violet and Bobby, the red-tail hawks at NYU.  Watching Violet continue to sit after all experts said that the egg wasn't viable was heart-touching.  And when Pip hatched on Mother's Day, it was a "wow" moment.  And there were others:  the wonderful Decorah, Iowa eagles next, a hummingbird in Hawaii, and the Roosevelt Arch camera from the Yellowstone Association.  I am so amazed that I can go to the computer and watch wildlife, live, up-close and in natural settings.

Young people and meaningful messages
Meaningful personal stories shared by young people using YouTube as the medium really touched me this year.  Zach Wahls and Ben Breedlove had a lot to say about life and living it well, with open hearts and open minds.

People and gadgets
This was the year that iPads really took off, and iPhones and Droid smart phones became pervasive around campus.  The willingness to carry a gadget everywhere you go surprises me.  The guru of understanding the connection of people and technology, Steve Jobs, passed this year.  The story of his life was like a walk down my own tech memory lane.
 

Holiday Break - Slowing Down

We started a tradition a few years ago that has become a part of our culture.  The last work day before the break is designated as "Cleaning Day."  We stop work on projects and we don't go to meetings.  We bring in coffee and snacks, and we provide pizza for lunch.  Staff are encouraged to dress casually.

We provide a variety of cleaning products and suggest that everyone pitch in and clean up.  This is physical as well as virtual.   The result is a winding down of activity to the break.  There's laughter as we find really old software diskettes or manuals.  There's a lot of "Why did we keep this?" talk.  And we leave the space really fresh for our return in the new year.

The list of suggested activities includes:

Update all assigned help and project tickets with current status and clearly described next steps.

Update any status reports.

Update the wiki or other documentation.

Clean up old email and reorganize your labels. 

Review and delete unneeded files from your desktop or laptop computer.

Delete and organize files on shares.

Reorganize your desk and drawers.

Backup your computer.

Clean out and drop in the shred bin paper files and documents from your desk.

Wipe the coffee and pop spills off your desk - we provide cleaning spray and paper towels.

Dust off the tops of cubes, window sills, window blinds.

Wipe off door knobs, your phone and your cell phone.

Join in the clean-up of common areas such book shelves, refrigerator, microwave, kitchen and such.

Note that the refrigerator and freezer will be emptied and anything left will be thrown in the trash.

Recycle old periodicals.

Clean up areas we'll identify, reorganize, put everything in its place.

Take time to review university emergency response materials here http://www.police.oakland.edu/emergencypreparedness/resources

Go through the short online training for Preventing Workplace Violence on campus:  http://www2.oakland.edu/training/cal.cfm?courseid=525

If you have any departmental rituals for your organization, please share.

Happy holidays!