The Braindonor Network

jQuery and ASP UpdatePanel Integration

John Hoff — Tue, 01 Jun 2010 17:42:10 +0000

With my latest gig sending me back into the .NET world, I have been busy migrating quite a few of my tools and experience to play nice with ASP.NET sites. One of my favorite open-source tools that is gaining popularity in .NET is jQuery. For several years now, I have been using jQuery heavily to provide rich interaction on websites. When faced with the basic UpdatePanel in ASP.NET, I was presented with a challenge. I wanted a solution that would automatically call jQuery so that new content was properly parsed and updated. Because I had been managing all the AJAX calls in my other development, this was trivially easy to do. UpdatePanel hides all of this inside the ASP AJAX libraries and scripts—making it difficult to figure out where and how to structure my jQuery code. After some research and experimentation, I have a solution up and running on multiple sites now that integrates everything in a very clean manner.

Scoping jQuery Rewrites
The first step is to scope all of the jQuery rewrites. The typical method of integrating jQuery rewrites into a website is to add a document.ready function like so:

$(document).ready(function() {
	$('.content_tab_button').click(function(){
		$('.content_tab_container').load('/newcontent.html');
	});
});

Because the rewrites are only called when the document.ready event fires, integrating with new content pulled in from AJAX needs to be done differently. By placing our rewrites in a new function that recieves the scope being updated, we can have a single initialization handle all of the updating:

function my_initializer(data){
	data.find('.content_tab_button').click(function(){
		$('.content_tab_container').load('/newcontent.html');\
		my_initializer($('.content_tab_container').html());
	});
}
$(document).ready(function() {
	my_initializer($(document));
});

Everything is now scoped cleanly. The rewrites are applied to the document when the page is loaded, and the new content when the tab button is clicked. Even if you are not an ASP.NET developer, please take the above scoping strategy with you!

How The UpdatePanel Works
At the highest level, an UpdatePanel refreshes its entire contents when a specificied event occurs within the panel. Server-side, the event triggering the refresh is processed, and the entire page-stae is regenerated. Then, only the content inside the UpdatePanel is pushed out. This is important for any developer using UpdatePanel to understand: UpdatePanel is much more convenient than it is efficient. As a result, our solution must follow that model.

With that in mind, we look to javascript class that is already managing the UpdatePanel, the Sys.WebForms.PageRequestManager class. In this class, we see an event handler for the pageLoad event. In the documentation, you can see that we are able to fetch the update panels that were just updated. In the example, they are looking for a panel with a specific ID. In our case, we are looking at having our rewrites applied to any panel that was updated:

var panels = args.get_panelsUpdated();
if (panels.length > 0) {
    for (i = 0; i < panels.length; i++) {
        my_initializer($(panels[i]));
    }
}

Because we have already scoped our jQuery code, this will work like a charm.

Putting It All Together
We can put all of the code together now, and we'll have a script that will play cleanly with the UpdatePanel:

function my_initializer(data){
	data.find('.content_tab_button').click(function(){
		$('.content_tab_container').load('/newcontent.html');
		my_initializer($('.content_tab_container').html());
	});
}
function my_pageloaded_initializer(sender, args) {
    var panels = args.get_panelsUpdated();
    if (panels.length > 0) {
        for (i = 0; i < panels.length; i++) {
            my_initializer($(panels[i]));
        }
    }
}
$(document).ready(function() {
	my_initializer($(document));
	Sys.WebForms.PageRequestManager.getInstance()
	    .add_pageLoaded(my_pageloaded_initializer);
});

All of our rewrites are called once—and only once—during page load and UpdatePanel refresh. Enjoy!

New Language Direction With Erlang

John Hoff — Thu, 11 Mar 2010 23:30:26 +0000

In my two most recent posts, Always Learning and Coders at Work, I talked about my personal and professional need to keep learning. My most recent addition to my development arsenal has been Python. Why I considered learning the language a success, I didn't really feel that it a significant amount of new materials and features. As a result, I have chosen to dive into the functional programming language Erlang.

I know that I need to address the issue of Python not adding enough new material to my utility belt before I get anybody riled up. Nearly all of the language features such as its primarily oriented paradigm, dynamic typing, and automatic memory management are not new--I also have experience and a high degree of comfort with C, C++, C#, JavaScript, Perl and PHP. Throughout my introduction to Python, I was simply able to look at how I performed certain tasks in the other languages, and ported things over without any real difficulty. I enjoyed learning Python immensely and I am using it more and more in place of Perl.

Now that I have gotten that disclaimer out of the way, I have to pull us back to the subject at hand: What made me choose Erlang? I wanted to learn something completely different. All of the languages I listed above are procedural languages, while Erlang is a functional language. Also, one of my favorite topics of study is scalable web architecture--yes, I study it even when I don't have to for work or a client. Erlang boasts a concurrent, distributed, fault-tolerant paradigm...all of which are issues that make scalable architecture so challenging and interesting. My final reason is that it isn't a large-scale popular language. Yeah, I'm the guy who will zig while everyone else zags and have a good time doing it. As I am only spending my own time and money in this endeavor, no clients will be harmed during the filming of this episode.

A lot of the blame for getting fired up for Erlang has to be leveled at the interview of Joe Armstrong in Coders at Work. Especially considering the first post I see on his blog dealing with websockets in Chrome. It all really started clicking at that point. Want a highly interactive web application? Have the browser on one end, and an Erlang process on the other end. Want thousands of other people interacting as well? No problem! The recent opening of Facebook Chat to everyone clearly showed off the power of Erlang through ejabberd.

So how am I going about this? Book and interpreter in hand, that's how! I'm still working through the exercises and very thankful that none of the recursion and induction exercises from college fell out of my head during the last 10 years. Next up is getting Yaws up and running on a server and start playing around with web apps. I should have a fun example or two in the next couple of weeks, so stay tuned.

Time for me to head off and go hack up some more Erlang code!

Book Review: Coders at Work

John Hoff — Wed, 24 Feb 2010 17:34:31 +0000

The book, Coders at Work was published at a very ideal time for me. I was pretty comfortable with my progress towards learning Python last year, and have been looking around for another language to start tooling around with on my computers. The book seemed like a good opportunity to get a peak into the minds of a select group of coders and see what they might have to say on the matter.

Coders at Work, by Peter Seibel, is a collection of interviews with fifteen very accomplished programmers and computer scientists. These individuals range from the young and self-educated to retired doctoral researchers. That was what first caught my eye on the book. It was very clear that with that broad range of contributors, I thought it would give a nice unbiased view on programming and computer science.

I read the book with a specific question in my mind and used it to measure what I read: What does this person have to say that I can use to be a better software developer? I hardly believe that I am alone in asking this question about my career and there is a wealth of material out there that aims to do that. As I read the book, I realized that I could ask a more important and specific question about myself: What does this person have to say about how I perceive and approach programming? After my natural tendency to analyze how we are different and similar has been satisfied, the next step was looking at how we both reached our current approaches. For me, this is where the interviews really shined.

Peter walked each of his subjects through a history of how they learned to program. He drew out the important programming accomplishments, motivations, mentors, classes, papers and books that shaped each of these programmers. At this point, I knew I had struck gold. I had in my hands a list of topics that I could explore. I could start running down the rabbit trails and see how I come out the other side.

Now that I have finished reading the book, I have even more research ahead of me and a sense of excitement. No, I'm not going to run off with the intent of memorizing the papers and mastering the tools presented in the book. Instead, I am going to continue to do what I did when I read the book: mine for more programming gold. To my fellow coders, I don't believe I can recommend the book enough. You may not take away nearly what I did, but I'm very sure that you will learn something about yourself.

New Category: Always Learning

John Hoff — Wed, 24 Feb 2010 17:03:16 +0000

One of the most apparent aspects of both my professional and personal life is that I am always learning. As my friends and family can tell you, I constantly have my nose in something and am very eager to share what I just learned. As I have written about what I have learned here, I have realized that I have been holding back on something important: how do I choose what I want to learn next.

When I was younger, I simply wanted to learn everything I could...and quite often the reason I learned something was because I could. As I have gotten older and my time has gotten more valuable, it is only natural for me to put more thought into what I learn. With a career, family, and friends it is not hard to see why. When I am able to decide that I want to learn something now, I always take a look at the utility of what it is I want to learn and the utility gained from learning it.

I still spend ample time learning about games, reading stories, cooking, and in general soaking up information--most of which is only useful if I'm allowed to play Trivial Pursuit. (I say "allowed", because once your family hears you call out an answer to a particle physics question from the next room, it's all downhill.) I am still very much a kid at heart. Something that entertains and interests me has an inherent utility that cannot be ignored. Crazy at it sounds, this means that I would consider learning Pokemon along with learning Python.

It is my hope that I can share these motivations more often. And yes, they will range from the silly to the technical. So check back with me from time to time and I'll see what gems and surprises I have to share.

Mako Authentication Required

John Hoff — Thu, 21 Jan 2010 21:11:55 +0000

I have been using my mod_python Mako handler for several months now in my personal projects. For the most part, I have been very happy with Mako and am finding it extremely useful. One issue I have had to wrap my head around has been the inability to halt template execution cleanly. A common practice in many a website has been to flush the current output buffer, display the required authentication information with a form or a redirect, and then end the request—making authentication required on a page of content.

Mako posed a significant challenge to my own thought process as I was designing page templates. I was very used to having the content buffered and being able to perform an internal redirect or transfer. Both mod_perl and ASP.NET offer this feature as part of the various frameworks. Luckily, my work in PHP has helped me pay much closer attention to how I am handling the internal flow of programs. Since most PHP configurations are set to be unbuffered, you have to pay particular attention to when and where you are delivering headers to the client browser.

One way to do solve this was to introduce exception handling to my mako handler. In ASP.NET, when an internal transfer or redirect is made an exception is thrown through the stack. In almost all cases, this exception can be cleanly ignored. To introduce this behavior, I would need to add acceptable exception conditions to my handler. Further, I would have to code my specific authentication methods into my generic handler. This was something I did not wish to do.

Instead of working with exceptions, I looked to the template inheritance of Mako to solve the problem. I start with a non-authenticated template which is in turn inherited by an authenticated template. The page requested inherits the template that corresponds to its need for authentication.

The following example pages illustrate how inheritance can be used to introduce the required authentication.

template.html

<%
    self.page_executed = False
%>

${next.body()}



Page Executed: ${self.page_executed}

require_auth_template.html

<%inherit file="template.html" />
<%!
    from my_library import Account
%>
<%
    self.account = Account.fetch_from_request(req)
%>
% if self.account is None:
    You must log in to continue.
% else:
    ${next.body()}
% endif

page1.html

<%inherit file="template.html" />
<%
    self.page_executed = True
%>
This page will always be displayed in the template.

page2.html

<%inherit file="require_auth_template.html" />
<%
    self.page_executed = True
%>
This page will only be displayed in the template
if the user has logged in.

Output of page1.html

This page will always be displayed in the template.

Page Executed: True

Output of page2.html

You must log in to continue.

Page Executed: False

Because the user is not logged in, the inheritance chain of page2.html ends the request and sends the login content to the top-level template without even calling the template of page2.html.

This technique can be further expanded to introduce exception handling by wrapping the next.body() call in a try block. This would keep all of the necessary exception and authentication logic outside of template handler and in the templates where they belong.

Custom Pages with WordPress Plugins

John Hoff — Tue, 18 Aug 2009 20:16:43 +0000

One of the more interesting challenges I have faced with WordPress is offering custom pages. These could be pages such taking a survey, asking a question, or suggesting a topic. Previously, I had gone about coding these kinds of pages by just sticking the php file in the web root. The problem with that strategy is twofold. The page cannot be disabled without going to the file system. The page also doesn't respect theme changes cleanly.

The solution comes in three parts. The first step is updating the rewrite rules to map a url to an internal value. Next, we map this internal value to a template. Finally, we map this internal template to the theme template.

Most WordPress installations take advantage of URL rewriting on the part of the web server. The goal is to rewrite all blog page requests to index.php. From there, WordPress examines the original URL to determine what content to send. The WordPress rewrite rules allow us to modify this URL. In our case, we want to map a custom page to query variable that we are going to pass into index.php, sending it down the $wp_query.

add_filter('generate_rewrite_rules', 'custom_page_generate_rewrite_rules');
function custom_page_generate_rewrite_rules($wp_rewrite) {
    $custom_page_rules = array(
        'survey' => 'index.php?custom_page=survey',
    );
    $wp_rewrite->rules = $custom_page_rules + $wp_rewrite->rules;
}

Now that we are able to map a url to the $wp_query, we need to resolve that into a template. For this, we utilize the template redirect of WordPress. This allows our plugin to resolve the request into a template call and exit before wordpress attempts to resolve the request.

add_action('template_redirect', 'custom_page_template_redirect');
fnction custom_page_template_redirect() {
    global $wp_query;
    $custom_page = $wp_query->query_vars['custom_page'];
    if ($custom_page == 'survey') {
        include(ABSPATH.'wp-content/plugins/custom_page/survey_page.php');
        exit;
    }
}

For the final piece, we will take a look at just one of the custom pages, survey_page.php. The template redirect hands off the request processing to this page and exits. It is up to this page to interact with the existing theme and display our custom page. The only catch is that the theme must utilize the the_content() template function of WordPress. Otherwise, the proper hooks will not be called and our page will not display correctly.

Fetching the proper file to use from the existing theme is very simple. To do so, we resolve the page template first, single post template second, and finally default to the index.

if (file_exists(TEMPLATEPATH.'/page.php')) {
    include(get_page_template());
}
elseif (file_exists(TEMPLATEPATH.'/single.php')) {
    include(get_single_template());
}
else {
    include(TEMPLATEPATH.'/index.php');
}

Updating the content involves inserting filters onto the_title and the_content. However, we only want these filters to be called during the_loop. Surpressing the filters from get_header, get_sidebar, and get_footer will prevent the custom filters from overrideing page template content.

add_action('get_header', 'custom_page_remove_filters');
add_action('get_sidebar', 'custom_page_remove_filters');
add_action('get_footer', 'custom_page_remove_filters');
function custom_page_remove_filters() {
    remove_filter('the_title', 'custom_page_title');
    remove_filter('the_content', 'custom_page_content');
}

add_action('loop_start', 'custom_page_add_filters');
function custom_page_add_filters() {
    add_filter('the_title', 'custom_page_title');
    add_filter('the_content', 'custom_page_content');
}

I went ahead and put all of this together into a plugin on this site. You can see it in action with the following pages:

I have also attached the source code for the plugin as a full example. You can download it here.

New Dish: Chili Lime Pork

John Hoff — Sun, 09 Aug 2009 17:49:45 +0000

After spending a good portion of the day with Chris and Xander out in the Chicago heat, I let them know that Dad would be taking care of dinner tonight. I remembered that we had a pork tenderloin roast in the freezer, so I got an idea. Little did I know just how good it would turn out!

On the left, you can see the wild creation I decided to make. Pork tenderloin strips beaten into fillets, marinated in Rose's Lime Juice, and finally grilled with a homemade sauce. Total time was about 2 1/2 hours, most of witch was spent defrosting and marinating. It turned out so well that I couldn't help taking a picture and share how I did it.

The first step was preparing the pork. I started out with a small pork tenderloin roast, which is typically two smaller tenderloins tied together or in a mesh bag. I have previously cut the pork length-wise before grilling, but did want the meal to take as long—it was well over 90 degrees outside. I decided to cut it into quarters and then tenderize it into fillets. I took each length, placed plastic wrap over the top, and beat the heck out of it. The final product was closer in shape to a fish fillet. With the new thickness, I knew that I could grill them in about 10 minutes and keep from being outside for too long. I then placed the fillets into a dish to soak in Rose's Lime Juice and several tablespoons of kosher salt for about an hour.

The seasoning starts out with three small limes. I zested and squeezed them into a small bowl, then added a touch of olive oil. To that I added salt, pepper, paprika, garlic powder, chili powder, and cilantro flakes. I kept things light with about a teaspoon of each. If you try this and end up with a paste, add more lemon juice and olive oil. You want the consistency of a thick vinaigrette.

With all of my preparation done, I fired up the grill. Once my coals were hot enough, I placed all of the pork on the grill. After about a minute, I flipped the pork and brushed on the seasoning. I had things hot enough to have very clear sear lines and the start of browning after only a minute. If you try this out on a gas grill, you will likely have to take a little more time. I flipped and brushed the pork a total of three more times. I continued to cook the pork until the surface was a nice golden-brown. When I thought it was finished, I checked one of the thicker slices with a knife to ensure that it was cooked through—which I highly advise people to do no matter their experience.

I retreated out of the oppressive heat to enjoy my new creation with my family. I took the picture shortly after coming in from outside, because I thought i had something special. After tasting, my wife wholeheartedly agreed. I can't wait to try this again!

Hijack the WordPress Media Gallery

John Hoff — Thu, 06 Aug 2009 21:05:57 +0000

I have a friend that wants to update a banner on his WordPress blog homepage without changing the template files. I tell him I'd be glad to help him out and start working on a plugin that provides a custom administration page to allow for this. As I am a firm believer in the power of a lazy coder to get things done faster and better, my first thought was: "Why don't I just see if I can hijack the 'Select Image From Gallery' page to do this?" After all, it just pops up into an Iframe page and tells post editer to insert an image. Hijacking that process for my own ends shouldn't take me but a few minutes...and a lot less time than trying to build my own media selector. Like nearly all the content on my site, if you are reading this, I wasn't able to find my answer elsewhere.

First, we will examine how the media library iframe is launched from the post editor and how we can duplicate that in our plugin. So lets crack open the page source and see what we can find:

Looks like a standard hyperlink, with some extra information in the tag. What first jumps out at me is the onclick="return false;". By having an onclick return false in a hyperlink, you ensure that a browser with javascript will never open the link when it is clicked. Sounds pretty silly until you remember that WordPress is a big user of JQuery. From there, we know to look at the class of the hyperlink, class="thickbox". This is a reference to the Thickbox 3.1 library found at /wp-includes/js/thickbox/thickbox.js. This is a library to load an iframe inside an overlay with the background greyed out. Further, it uses the href of the anchor tag to know where to pull the iframe content from.

A quick check of the source on our custom page shows that Thickbox is being loaded in the footer, so we do not have to call it directly from our custom wp-admin page. I can just call up the media library with some html code:


" width="450" height="25" />

The snippet displays an image with the source of $banner_image that can be clicked on to open the Media Gallery iframe. I have also added a hidden form input to store the resulting information passed back from the media library. Once done, I now have an image on my admin page that I can click on to call up the media library.

We're half-way there now! All we have to do next is figure out how the media library sends an image to the post editor. This part isn't as easy. Thanks to thickbox, the media library opens up in an inline-iframe. This means that we can't just view the page source and find our answer. Instead, the only clue we have to hunt for is the "Insert into Post" button. A quick file search turns that up in /wp-admin/includes/media.php. From there, we discover the onclick of the button, onclick="addExtImage.insert();", which is thankfully also in the media.php file. At the end of the function, we see exactly what we are looking for:

var win = window.dialogArguments || opener || parent || top;
win.send_to_editor(html);

The media library calls the function send_to_editor in the scope of the window that opened the media library--our admin page! We can add our own function into the admin page and collect the html that the media library sends us:

function send_to_editor(html) {
    alert(html);
    tb_remove();
}

Once done, we can open the media library from our admin page and select an image. When we hit the "Insert into Post" button, we get an alert with the following content:

All we have to do now, is strip out everything but the image source and updates the form. First, the function eliminates all of the text returned except the image source. It then updates the banner image and the hidden input value that I am using in my form:

function send_to_editor(html) {
    var source = html.match(/src=\".*\" alt/);
    source = source[0].replace(/^src=\"/, "").replace(/" alt$/, "");
    $("#banner_image").attr('src', source);
    $('#banner_image_input").attr('value', source);
    tb_remove();
}

I fire up the admin page a final time, and everything works! I now have a custom admin page that allows my friend to update a banner on his wordpress site using the media gallery.

Apache Optimization and NGINX

John Hoff — Wed, 29 Jul 2009 18:50:51 +0000

Once again, my WordPress friends had to hit me up for some help. These are the same friends that I helped with a Custom Field Search Plugin. They have been maintaining a community site, SuccessNet Online™, and an email mailing list of several hundred thousand participants for Business Networking International. They send out an email every month with a quick update on the community and directing participants to visit the site. For several months they had been using a Lyris server in their office to send out the mailing list. This past month, they started using Lyris HQ to send out the newsletter. When the first newsletter was sent out using the Lyris HQ, the community site quickly ground to a halt and nearly crashed the server. They quickly gave me a call to figure out why the site was having problems and what we could do to prevent it from happening again.

The reason for the site performance issues was immediately apparent once they told me that they were no longer using the Lyris server in their office. They went from an 3-year old server sitting behind a T1 to a managed service sitting behind who knows how big of a pipe. Instead of taking 7-8 hours to deliver the email newsletter using the office server, the new service was able to do it in about 15 minutes! As a result, the peak traffic to the site was compressed from 12 hours to a single hour. Using the default Apache configuration, there was no way they could handle the traffic. With about an hour of work, I was able to optimize their server configuration to handle the traffic. Because I was able to do this without increasing the resource footprint of their server, my friends were very happy.

This same story is repeated time-and-again by businesses large and small that encounter sudden spikes of traffic. The reason for the spike can range from a Slashdotting to the removal of a resource bottleneck. The result is nearly always the same and exposes a fundamental weakness in common Apache configurations.

Anatomy of an Apache Failure
The default configuration of Apache is set up to handle the delivery of static files. This default configuration is able to deliver massive amounts of content within a small footprint. When you begin adding an application layer and a database to the mix, you introduce the ability for Apache to bring even high-end servers to their knees.

Most web applications that run under Apache require the use of the PMP prefork configuration. These include web applications written in PHP, mod_perl, mod_python, etc. The critical section of the Apache configuration that we are going to look follows:


    StartServers          5
    MinSpareServers       5
    MaxSpareServers      10
    MaxClients          150
    MaxRequestsPerChild   0

When this configuration is placed under load, Apache will create up to 150 worker processes to handle the requests. A typical worker process that handles a WordPress site will weigh in at about 30MB of memory. In order for a web server to handle 150 of these workers, it would need at least 4GB of memory! This is also a very optimistic estimate. I've seen quite a few site applications that end up with workers using 200MB of memory and up. The typical web server has no where near enough memory to handle 150 workers.

At idle, this Apache configuration will have ten idle worker processes. Now we start adding traffic, measured in concurrent requests. Once there are 6 concurrent requests, Apache creates an additional worker because it is required to have a minimum of 5 spare workers. The creation of a new worker process is not free and it does have an impact on the load of the server. As load continues to increase, Apache will continue to create worker processes. Once all physical memory is exhausted, additional workers will begin to hit swap memory. The overhead required to use the swap memory imparts even more load on the system. Once this point is reached, worker creation will snowball out of control. One of two things will typically happen—and happen fast: the server enters swap-death; or the database crashes under too many connections.

We have all seen both on many occasions. In the case of swap-death, the browser connection times out. In the case of a crashed database, we get the standard 'unable to connect to database' error. So how do we avoid that happening on our own site?

Strict Apache Resource Limits
The only sure-fire method to preventing load from disabling the web server is to configure Apache so that it will not consume more resources than are available. In the case of the WordPress site I was called in to help on, the server had 1GB of memory and was running both Apache and MySQL. My plan called for having Apache consume no more than 2/3 of the physical memory of the system, leaving the remaining 1/3 for MySQL and the OS. Using the previous 30MB estimate, this allows for a maximum of 20 worker processes:


    StartServers          10
    MinSpareServers       10
    MaxSpareServers       10
    MaxClients            20
    MaxRequestsPerChild    0

As you can see, I also increased the start, min, and max number of processes. I did this to further reduce the frequency of worker creation and destruction—reducing overhead as load increases and decreases. For high traffic sites that utilize a dedicated database server, I recommend setting the start, min, and max servers to max clients.

With the changes made, it is time to test things out. My personal preference is to fire up several looping wget scripts that also download the images, javascript, and stylesheets for the page:

#!/bin/bash
while true; do
wget -pq http://www.server.com/
done

With the resource limits in place, the system load will remain at acceptable levels even with several scripts generating traffic. However, as the number of traffic scripts are increased, the responsiveness of Apache decreases at an alarming rate. When I used this method on the WordPress site, I was experiencing delays with as few as two scripts generating traffic.

Introducing a Reverse Proxy
Now we have Apache in a configuration that no longer runs away with our resources, but the site appears to be significantly slower to respond to requests. The solution is to have separate web server instance to serve all static content. The easiest way to accomplish this is through the use of a reverse proxy. The reverse proxy responds to all web requests to the site. If the request is for static content, it delivers the content. If the request is for dynamic content, it requests the dynamic content from Apache. This ensures that the heavyweight Apache worker processes are only being used to respond to dynamic content requests and a lightweight process is being used to respond to static requests. Otherwise, the heavyweight workers will spend most of their time responding to static requests.

I cannot stress the importance of a reverse proxy enough for high volume sites! Without a reverse proxy, our small pool of heavyweight processes spend a significant amount of time just sending the raw data for the static content. By having a separate web server send the data for static content, you minimize the impact that data delivery will have in tying up worker processes.

My server of choice for the role of reverse proxy is NGINX. NGINX is easy to configure, fast, and stable. The first step of installation is to reconfigure Apache to listen to a port other than 80. Port 8000 or 8080 are commonly used. Once that change has been made and confirmed, NGINX can be installed.

I prefer to set up NGINX initially to proxy all content from port 80 to port 8000. This allows me to verify that everything is working correctly before I set up the static handlers. I then add a proxy for the specific hosts that I need. An edited version of the configuration that I use on this site follows:

nginx.conf

user                    www-data;
worker_processes        5;
error_log               /var/log/nginx/error.log;
pid                     /var/run/nginx.pid;
worker_rlimit_nofile    8196;

events {
    worker_connections    1024;
}

http {
    include         /usr/local/nginx/conf/mime.types;
    include         /usr/local/nginx/conf/proxy.conf;
    default_type    application/octet-stream;

    access_log  /var/log/nginx/access.log;

    sendfile        on;
    tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;
    tcp_nodelay        on;

    gzip  on;

    include /usr/local/nginx/conf/sites/*;
}

proxy.conf

proxy_redirect          off;
proxy_set_header        Host            $host;
proxy_set_header        X-Real-IP       $remote_addr;
proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size    10m;
client_body_buffer_size 128k;
proxy_connect_timeout   90;
proxy_send_timeout      90;
proxy_read_timeout      90;
proxy_buffers           32 4k;

sites/000-default:

server {
    listen 80;
    server_name _;

    location / {
        proxy_pass  http://127.0.0.1:8000;
    }
}

sites/braindonor.net:

server {
    listen 80;
    server_name braindonor.net *.braindonor.net;

    # serve static files directly
    location ~* ^.+.(jpg|jpeg|gif|css|png|js|ico)$ {
        root /var/www/braindonor.net/htdocs/;
    }

    # proxy the rest
    location / {
        proxy_pass  http://127.0.0.1:8000;
    }
}

Be sure to check both the Apache and NGINX log files to ensure that requests are being handled by the appropriate server.

Wrapping Things Up
With the reverse proxy in place and running, there are a few tasks to wrap up.
The LogFormat in apache.conf needs to be updated to reflect the use of a reverse proxy:

LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b" common

If you are using any log analysis programs, you will need to update them to use the logs from NGINX instead of Apache.



Don't hesitate to leave a comment if you have questions or need some help!



Custom Field Searching WordPress Using Sphinx
John Hoff — Wed, 20 May 2009 19:15:59 +0000
In my previous post on implementing a custom field search in WordPress, I showed how to modify the internal SQL LIKE search that is the default search engine of WordPress.  There are other search engines that WordPress can make use of, most notably Sphinx Search.



Sphinx is a great open-source full-text search engine.  I have used it several times to implement search functionality in content management systems.  I didn't go into using Sphinx with WordPress initially because it is a lot more work.  The default WordPress search is an SQL query performed by PHP.  Sphinx is composed of an indexer, a search daemon, and a client library and the installation of the package is unavailable in shared-hosting environments.



Since more and more people are using WordPress on dedicated servers, I'm not surprised that questions on implementing custom field searching using Sphinx has arrisen.  The good news is that it is very easy to implement!  All you have to do is update the sphinx.conf to pull out the custom fields as fields in the sql_query of your sources.



In my previous example, I worked with the fields: bio, byline, kicker, and deck.  I'll use these same fields again for continuity sake. I'm also using those fields because I have a site that uses these fields for custom search and I could test the changes needed for Sphinx.



In order to add each field, we have to do three things.  Add the column to the select on the wp_post, add a left join to the select on the wp_post, and add a placeholder to the select on wp_comments.



Example Columns:

p.post_content as body, \
t.name as category, \
bio_meta.meta_value as bio, \
byline_meta.meta_value as byline, \
kicker_meta.meta_value as kicker, \
deck_meta.meta_value as deck, \
IF(p.post_type = 'post', 1, 0) as isPost, \
0 as isComment, \


Example Joins:

inner join \
{wp_terms} t on (tt.term_id = t.term_id) \
left join \
{wp_postmeta} bio_meta on (p.ID = bio_meta.post_id and bio_meta.meta_key = 'bio') \
left join \
{wp_postmeta} byline_meta on (p.ID = byline_meta.post_id and byline_meta.meta_key = 'byline') \
left join \
{wp_postmeta} kicker_meta on (p.ID = kicker_meta.post_id and kicker_meta.meta_key = 'kicker') \
left join \
{wp_postmeta} deck_meta on (p.ID = deck_meta.post_id and deck_meta.meta_key = 'deck') \
where \


Example Placeholders:

c.comment_content as body, \
'' as category, \
'' as bio, \
'' as byline, \
'' as kicker, \
'' as deck, \
0 as isPost, \


I have also attached the sphinx.conf distributed with the WordPress plugin so that you can see a more complete example of the implementation.



I hope everyone enjoys this as much as they did the last!