Web Site
Developers
Dynamic Content - News Summary Page - II
A Better Way
Again inspired by Phil Howard's [1.], whose penchant for seeing simple text files as the solution, I have found, indeed, it would have provided the quicker, less error prone approach for updating the News page on the defunct OpenSourceToday site. Moreover, not only would have it been, it is simpler; it also had the additional advantage of the ability to discard older, rotted stories before they become an obvious embarrassment. That is what I will show here, the more direct method to update these pages relying more heavily upon text files, while reducing the risk due to excessive processing.
Basic Assumptions
Not too much different than I outlined in the previous two articles [2.] in this series. Previously, I tended to just say the temporary storage questions were unresolved, hence, let's not worry about it. I have even more reason to push that view now.
News Item Temporary Storage
I would like to have the reader remember the News item inputs were more limited. Indeed, I now think the title field could be skipped with no harm. With more fields by default left blank, our security issues are lessened. I think too, that particularly for the OpenSourceToday site, I could have insisted on a fixed form for submission. This is what a news item would have been in a temporary file:
<h2 id="central-text">Open Source Security \
and Data Center Consolidation become \
Key Opportunities for Resellers</h2>
<p>A study by the Federal Open Source Alliance \
has revealed that the Federal Government has \
an increasing appetite for open source. ...</p>
<p>Similar to the state government market, the \
Federal agencies data center consolidation is \
... download at this web site.</p>
<p>Surveys of the Department of Defense (DoD), \
... </p>
Listing 1. News Item, as Stored in
Temporary File
In addition, the file would be located away from the server directories with very restricted access rights. Nonetheless, this file's content is ready for use upon verification. Moreover, it needs little processing to be useful content in the News Page, even using the older methods. However, that was the last article's approach, not what I plan to offer here.
Begin News Item File as Batch Process
Once the input in the temporary file has been vetted, a new script has to go into action. The first step is to copy the temporary file into the production directory. I think, however, that the reader should be warned. This step is NOT part of the page rendering, that happens at most once per news item. I advise seeing this as a process more akin to a batch process that performs initial preparatory steps before the the use can commence.
Methods to Copy File into Production Directory
By design and necessity, I am conveniently ignoring significant aspects implicit in this process. My defense is simple, these are thought experiments that I will never use. Hence, it is reasonable for me to limit my level of exertion.
The code could be run manually with an argument feed in to identify the file to be copied or it might be automated with more effort:
// file location & name where
// $path and $news are the
// off server directory and
// news item file name, resp.
$news_temp = $path . "/" . $news . "-" . $digital_time_stamp . "\.txt";
// where [path] is directory where
// working news items are stored.
$news_work = $text . [path] . $news . "-" . $digital_time_stamp . "\.txt";
// archive version (leaving out
// opening and closing of files
copy($news_temp, $news_work);
// remember to test for failures
Listing 2. Copy Temp File News Item
into Production Dir.
Prepare Date/Time Ordered Files in Production Directory
Before we go too far, I have some words of warning: the reader should see the displayed code as suggestions that might come close to be workable [3.], not tested, vetted and known to be good. Indeed, had I pursued the objective, I might have written parts in perl [4.] or more likely python [5.] rather than php [6.]. This will be a multistep operation where one of the listed scripting tools (or even java) might be the better option. Whatever the case, let's begin taking the ordered listing of these files in date, time order (latest at the top), which is dumped in a standard file. Then a line count is taken of that mentioned standard file, for possible later use:
// all in production directory
$order_news = $text . [path] . $news . "-\*\.txt";
$order_command = "ls -lt " . $order_news;
$store_in_file = $text . [path] . "store-list\.hold";
exec($order_command > $store_in_file);
wc -l $store_in_file > count_lines;
$num_lines = explode(" ", count_lines);
// number of lines
$total_lines = $num_lines[0];
if ($total_lines == 0) {
// report failure, cease processing
} else {
// continue, see listing 4.
}
Listing 3. Stored, Ordered Directory
List and Total Count
We are still in the initial steps of this batch process. However, I think it is time to reflect upon the needs and goals before proceeding. The $total_lines gives an upper limit of the size of news items that could be read into the News page. It would have taken some experimentation, but I would have begun by displaying no more than five news items. Moreover, I would have looked closely at the date, time stamps on the individual files thinking that any number exceeding ten would be safe for auto deletion. Those were my presumptions, however, experience might have dictated different choices.
Latest File News Items in Date/Time Order Created in Production Directory
In the model I am presenting, the rendering depends upon a file that lists the file names in date, time order. That is the next step in this batch process. As I indicated above not every line has to be processed, the last variable can be used to simplify the code. Using the approximate numbers mentioned:
// open ordered list
$file_handle = fopen($store_in_file, "r");
// open new file, with listed file names
$render_list = $text . [path] . $news . "-list\.hold";
$file_hd_list = fopen($render_list, "w");
// not using this code, but could be used to
// both limit the size of file name list and
// to allow culling of excess, old files
if ($total_lines > 1 and $total_lines < 11) {
// no culling excess files
} elseif ($total_lines > 11) {
// set variable to allow
// deletion of excess news items
| else {
// error condition
}
// create new order news item list
while (! feof($file_handle)) {
$read_line = fgets($file_handle);
if (egrep '^\-' $read_line) {
$news_array = explode(' ',$read_line);
// not tested, thought * or + needed
// for multiple spaces, however, now
// suspect above suffices.
$news_item = $new_array[7];
fwrite($file_hd_list,$news_item);
} else {
// do noting or restart loop
// not a file name, skip
}
}
// close files, upon error stop
// and summon help.
Listing 4. Filling News Lists with
Ordered File Names
Copying Latest News Items File Over Last Production Version
We are still firmly within the batch process, however the next steps are common to those described earlier using different methods [7.] [8.] that had the same goal, but were preformed as part of the script that rendered the page. To limit duplication, I will just outline the steps, since these same operations are in the last set of citations. The big difference is, these will be executed in the shell. The steps are: create a backup of the file that handles the page rendering, copy the new (temporary copy) over the rendering file. If there is a failure in the last step an attempt is made to copy in the backup. However, the maintainers are notified of the problem immediately.
We begin with the backup:
// create identity for backup
$render_list_act_bkup = $text . [path] . $news . "-list\.bkup";
$render_list_act = $text . [path] . $news . "-list
$bkup_command = "cp -a " . $render_list . " " $render_list_act_bkup;
exec($bkup_command); // would over write existing file
// copy temporary list over older rendering file list
$copy_new_list = "cp -a " . $render_list . " " . $render_list_act;
exec($copy_new_list);
Listing 5. Completing News Item List
File Copying
That might give the appearance of some heavy duty processing, however, I will point out this all could be performed using a shell batch file [9.]that accepts a parameter identifying the file that is already stored in the desired format. The other steps were merely taking a file listing of extant files in a specified directory with a common naming, those names were deposited in another file a line at a time. Finally that newest file file was copied over the original after a backup step was run. Moreover, it was only that last copying that could have impacted the page rendering, all other potential errors would be maintainer harassment not loss of web site functionality.
Easier Web Page Rendering with Files
First a quick reminder what the core rendering code is comprised:
<?php
/* confirm file exist, then read */
readfile($[filename]);
?>
Listing 6. Bare Bones Rendering Code
where the comment refers to the error catching code. Remember too, however, that these files had to be constructed using multiple steps. In this model, there are now two sets of files, the ones with content and a second that only is a list of file names in date / time order. It was only the latter that required processing, but comparatively modest. We only built an ordered list of news items from a file name listing. The content files were present at creation, needing only to be copied into the production directory upon confirmation the contents were real.
The steps required to turn the code above into a program script that will work for the News page is a loop structure, that ceases at a pre-fixed maximum value or simply reaching the end of the file. I would suggest that the maximum number of stories be passed in as a parameter that can be changed as the best value is determined:
...
$news_handle = fopen($render_list_act, "r");
$news_number = 0;
$max_number = 5; // arbitrary choice
while (! feof($news_handle) and ($news_number < $max_number)) {
$news_number++;
$file_name = fgets($news_handle);
$News_item_handle = fopen($file_name, "r");
// each read file writes a story on the page
readfile($file_name);
fclose($News_item_handle);
}
// the News Page has been rendered
fclose($news_handle);
Listing 7. News Page Rendering Code
Done. Superficially this might appear as more complex, however, it is less error prone than the method used previously. Content files were subjected to minimal processing. Moreover, assume one file has been corrupted and can not be opened. One story might then be missing from the page, considering the probable content the loss in the larger scheme would be acceptable. This is the simpler, more effective program that I would have used on the OpenSourceToday site.
Summary
As I promised, I have shown an simpler, less dangerous method to use files to render dynamic content on web pages. In this particular instances, it would have worked on the News Page of the defunct OpenSourceToday site. However, with judicious choice of altered code the same process would work to program other site's dynamic content without excessive processing of the new and old content to show the newest results. The content all resides in separate files that are loaded into the production directory, only the date /time ordered file names need to be created and updated into a new file. That latter file is just read sequentially to pull the proper file names that are then opened to rendered the html formatted content. Very little in this process is open to catastrophic failure. By concept and execution the simplest of methods are employed. Conceptually simple, minimal exposure to process failure and minimal code comprises a resilient program.
I would strongly advise restricting automated culling of the excess content files to the batch process, just for safety concerns. Moreover, I think it would be wise to retained a few beyond the supposed maximum number as a buffer. Should one or more recent files be lost or corrupted, the News page will not be left empty. Whatever the case, little is lost retaining a safety margin.
Another suggestion I would follow is to avoid adding overly sophisticated code to detect errors too early in the process. I think, the contributor should play a role, because they have the most incentive. Indeed, the human eye is a better error catcher than the most sophisticated coded routines. Their presence also lessens the need for close watching by site maintainers.
What's Next?
I have beaten the life out of using the html input form I created for the old OpenSourceToday site. Probably most readers have tired too of this exercise, as have I. Therefore, I see little need to go beyond a limited discussion of the temporary files I would have used for this particular news item application. That will wrap this series as I will turn my attention to more real, pressing coding issues that apply to real, extant sites.
Corrections, suggested extension or comments write: H. Cohen.
© Herschel Cohen, All Rights Reserved
Return B/ST Home or Dynamic Menu Page
____________________________________________________________________
1. Owner, operator of Linux Home Page site. Return
2. How the News Items and the Article Summary Listing
were done , the hard way. Return
3. This is my practice, however, first passes rarely work
as expected. Nonetheless, in most cases, the ideas are valid.
Return
4. I prefer using arrays, perl was very weak when I last
used it version 5.5 or 5.6. However, the language excels
at string handling. Moreover, the people on the maillists
are the best I have encountered.
[Example, beginners-cgi@perl.org] Return
5. I am biased in favor of python's due to its design
and its implicit default of formatted code. However,
despite its robust nature and admired syntax, I am not as
comfortable with it as I am with php. Return
6. Personal preference, comfortable with the syntax, it's
like C (which I never coded professionally/seriously) and
answers come quickly. Return
7. Look here at the small section of code beneath Listing 4.
Same process. Return
8. Examine the code in the subsection:
Copy Temporary Over Original File to see the same
operations. Return
1. Shell batch writing is not my forte, since I have the
impression, the syntax is at odds with the scripting
languages I prefer. Return
____________________________________________________________________