|
Storing information - I As a software designer and developer, you have to constantly manage the storage and use of information. This information may be transient or permanent. It may even have a more complex persistence like transient storage on multiple locations, or store and forward existence like in an e-mail message.
Information, when looked at from the point of view of a programmer, is ultimately some type of data. The data may be of various kinds like integer, strings, image, date, time or a combination of these. This data is handled in a predictable and universal manner in most cases.
The most common form of storing and using data is from a database. A database typically is storage that is persistent. Persistent here means that you can retrieve the data even after the computer is shut down and then restarted.
Databases, as you are already aware, come in many shapes and sizes and provide simple to highly complex functionality to manipulate data and information.
In a typical client-server application, you will have one data server and multiple client computers. The data server stores all the data in a single database using the chosen data engine like SQL Server, Oracle and so on.
From the point of view of an application designer, however, a database may not be the only place where you store data.
It is not only the base data that is important but also other types of data that are to be stored, either temporarily or permanently.
Examples of other types of data are configuration settings, network parameters, security settings, user preferences, default values for various operations, local configuration of printer or monitor, and so on.
Thus, apart from the default database you require many other types of locations where you would need to store information of different types. With newer types of application development methodologies coming into the picture, like Web programming, three-tier programming, collaborative computing, etc, information storage requirements are becoming more and more complex.
In addition, due to the increasingly flexible architecture of the operating system, Web tools and development tools, many more diverse containers are now available to store different kinds of data at runtime.
Therefore, software designers need to be well versed with the existence and appropriate utilisation of these containers. Using the right method of storing data for the right kind of requirement is very important. Improper usage may hamper the security, performance, maintainability or usability of the application that you are designing.
Where do you need to store data?
There are many locations and facilities that allow you to store data, with varied persistence characteristics. Given below is a list of the more commonly used containers or facilities where data can be stored. This list is by no means complete.
This list started with just around 8 different non-database methods of information storage in mind initially. On exploring the subject further, the list just kept on growing. A brainstorming session with my development team finally led to a list of over 50 different methods of storing data that applications use in places other than your default database--and I am still sure that it is far from complete.
Storage methods:
Transient storage
1. Memory-based storage
2. RAM / temporary workspace
3. IIS runtime objects
4. Shared property manager
5. Property bags for ActiveX controls
6. Performance monitor counters
7. Keyboard buffer
8. Video buffer (Direct Draw)
Mixed storage
9. External storage devices
10. Memory database (COM+)
11. Tagging using XML
12. Message queue
13. Registry
Persistent storage
14. Local variables in documents
15. Mail message
16. Local text file (.INI file)
17. Shared directory with ASCII file
18. Registry on a central server
19. Heterogeneous databases
20. Spreadsheets
21. External devices
22. Common configuration file
23. Mailbox custom properties
24. VBA related containers
25. Cookies
26. ADTG files of ADO
27. Active Directory
28. Public folder
29. Hidden mail messages
30. File properties
31. Eventlog Windows API interface
32. Persistable objects
33. Templates for Office
34. Hidden elements in HTML code
35. URL file with parameters
36. Custom wizards
37. IE5 Persistance object
38. Offline folder for browser
39. External locations on web service providers
40. Repository
41. BIOS
42. Microsoft Wallet
43. Watches/electronic diaries
44. Handheld data acquisition devices
45. Voicemail storage
46. Font files
47. Windows CE devices with onboard storage
48. PST (Personal folders)
49. Online updation of help content
50. Dictionary object of VB 6
51. Resource files
You may think that it is unnecessary to know details of each of the above methods of storing information, as one typically requires only a few common ones like text files, registry and the runtime environment. However, as a designer or developer of applications, it is important to know which method to use for what purpose. This article seeks to provide this information.
One caveat though--what follows is by no means an exhaustive description. There may be more methods as well as more ways of using the methods described below. You'll need to use your imagination to write more innovative and relevant code as you design your applications.
Memory-based storage
This is probably the most common method. Every application uses memory variables and various other structures in memory for manipulating information. Memory variables are typically destroyed when you restart the computer.
Runtime vs design time manipulation:
Most application development environments allow you to have memory-based data items within their runtime environment. One important distinction about the variables is whether these are available for change while designing the application. Design time variables or properties include things like visual element characteristics. Some variables are available only at runtime. For example, the executable path is a runtime variable. System properties like the current size of the screen, current default printer and so on are memory variables which are accessible only at runtime.
Read/write vs read-only:
Some runtime variables are read-only. These are typically those pieces of information that are controlled and manipulated by the base operating system. For example, a list of available disks is a read-only list. You cannot programmatically change it because it depends upon the physical presence of the disks.
On the other hand, information like the path where you want the user to save a report may be read/write in nature.
Saving memory variables:
There are some languages that allow you the powerful capability of storing the current content of memory variables at runtime. Take, for example, the FoxPro command:
m.name = "sample"
m.age = 44
Save to mymemory
This command saves the contents of all the memory variables to a file called mymemory.mem. When at a later date, you want to restore the memory variables, you can issue the command:
Restore from mymemory
Dumping the memory:
When the Windows NT operating system suffers from a fatal error at the kernel level, it stores the exact contents of its entire memory to a disk file. This file is called a memory dump. This can later be used for debugging and analysing the cause of the crash. The memory dump size will be same as the size of the physical RAM on the computer.
RAMdisk
This was once a popular method of storing various types of information, including lookup tables from a database itself. This method relies on creating a dummy hard disk within the RAM itself. This disk is accessible just like any other hard disk. The only difference here is that its contents are completely wiped off when the machine power is reset. Thus a RAMdisk is used for lookup tables or non-critical data which is infrequently updated and requires instant access. The amount of available memory must be sufficient to accommodate the desired RAMdisk size.
RAM / temporary workspace
Sometimes RAM can be used for storing data in forms other than variables. The TEMPDB database of the SQL Server system can be created entirely in the RAM provided adequate memory is available. TEMPDB is used by SQL Server for creating interim data, queries, indexes and other information. It is anyway recreated every time SQL Server starts. Putting TEMPDB in RAM provides a very significant performance boost if the application utilises TEMPDB to a substantial extent.
|
