Posted March 29th, 2011 10:31:45 am by Jarrod Goddard
Filed in Linux, Web Hosting & Server Administration, Web Site Design & Development
We recently made some changes to our internal development server, all our client and business files have been moved off the actual web server and into a NAS (Network Attached Storage). Once we made this change, however, we found that none of our web sites were showing images correctly when browsed through a web browser. They looked fine in Windows Explorer or Mac Finder, but when accessed through a browser the images were not showing up at all.
We had just migrated about 300GB of data to the NAS from our Debian linux server and I thought maybe the file migration had an issue, or somehow the files became corrupt. But the files worked fine in Explorer or Finder windows. We also found that all the text files were ok, and just images were having the issue when referenced through a web browser. I created an image in photoshop and saved it to a folder on the NAS and then saved the same image to a folder on the server’s filesystem. The image showed fine in Explorer/Finder, and then it showed fine through a browser when referencing the file system version, but when referencing the image through a browser on the network storage it wouldn’t show up right. So now I knew the issue was with Apache, Images and network storages… from here, naturally, we go to google to find the solution because we can’t be the only ones who’ve had this issue.
So we googled “apache image truncate“, “apache network storage images” and a couple other variations and finally after searching 5 or 6 times we found a single other person who has had this issue on ServerFault (http://serverfault.com/questions/96460/apache-returns-truncated-image).
Reading that post, I learned about the EnableMMap and EnableSendFile options in Apache, and then referenced this page (http://httpd.apache.org/docs/2.2/mod/core.html) for more information. From the description on Memory Mapping:
This memory-mapping sometimes yields a performance improvement. But in some environments, it is better to disable the memory-mapping to prevent operational problems
One of the memory-mapping issues it then refers to:
Deleting or truncating a file while httpd has it memory-mapped can cause httpd to crash with a segmentation fault.
I’m assuming that was the issue we were having, so in our /etc/apache2/apache2.conf file (we’re running Debian) I added:
EnableMMap Off
The other setting that the ServerFault article referenced was EnableSendFile.
This sendfile mechanism avoids separate read and send operations, and buffer allocations. But on some platforms or within some filesystems, it is better to disable this feature to avoid operational problems:
…
With a network-mounted DocumentRoot (e.g., NFS or SMB), the kernel may be unable to serve the network file through its own cache.
So then I added the EnableSendFile Off option to our /etc/apache2/apache2.conf file, and restarted apache. Boom! Our images were showing up fine.
Anytime we have these kind of issues, where we are totally confused and need to google for more information, I like to blog about it so we have our own reference and, of course, to help everyone else who has the same issue, hopefully this helps.
Comments »
Posted July 14th, 2008 10:04:57 pm by Jarrod Goddard
Filed in Business, Productivity & Freelancing
I just got a new Mac Book Pro a couple weeks ago, and although I love it, the biggest adjustment I had to make was getting used to the keyboard. It’s missing a bunch of keys like the side number pad (most laptops don’t have these), but it doesn’t have a page up/down button, the delete button works like backspace and not delete, and there is no home or end buttons. This really bothered me for a while, until I found out their shortcuts.
All laptops have a ‘fn’ button that is used as a modifier key for the f1…f12 buttons and usually some of the other buttons on the laptop to provide the same functionality of a full keyboard but with less keys, and they’re labelled with a different color to indicate what those buttons are. On the Mac Book Pro they aren’t. Here they are:
- Home: <command> + arrow key up
- End: <command> + arrow key down
- Page Up: <option>+arrow key up
- Page Down: <option>+arrow key down
- Delete: <fn> + delete
It was a big relief to finally find these shortcut keys, I’ve been frustrating myself trying to adjust to not having them.
3 Comments »
Posted May 11th, 2008 01:36:10 pm by Jarrod Goddard
Filed in Web Site Design & Development
I was recently working on a custom content management system for a client and I was adding a search box to the site to allow users to search the client’s site. Because all the content is contained within a database, I can easily search through the pages table to grab any pages that have the search terms contained within their title, content, or keywords meta tag. I wanted to make it a little more robust than simply doing a select statement and grabbing any pages with the search term and I wanted to also rank the results based on a relevance factor determined by where the search words were found.
I was recently putting together a custom search form for a client, and wanted to improve on it from the basic “SELECT title FROM page WHERE title LIKE ‘%search-query%’” method of searching a site. I wanted to include some more relevance to the search so pages that have the search words in their title were a little more relevant, as well as pages with the search term in the filename, and keywords meta tag as well as the body.
I hate most search forms on web sites simply because they’re so horrible. Many of the sites I’ve worked on with pre-existing search components have just done a simple select statement that grabs any page from the database that has the search word in the body of the content. They don’t take into account the page title, keywords, filename, along with the content and it can often make for less-than-intelligent search results. This has even led to many site owners installing a google search box on their site, however, for the kind of projects I take on, I like to custom-build as much as I can for design flexibility and future maintainability.
Search is Important
Search is also very overlooked by site owners. They’ll setup something like google, or atomz, or htDig, and not really test the relevance of the search to see if the search results as relevant as they should be. Even google’s custom site search isn’t the most relevant when you’re site isn’t optimized accordingly, and when you’re clients are in control of the content you can’t ensure that the site is optimized for google – making even google site search less than favorable.
The search function in an e-commerce site is even more important than a content site as you’ll find that the search box is the primary method for users to find product on your site. That’s why Amazon’s search is so prominent on their web site.
Relevancy-based Search
Just as google considers so many factors in their search results, you should consider more than just the content of the page. In the case of my custom CMS, I’m allowing users to add keywords, descriptions, titles and other standard META tags and I’ll base my search results off each of those fields as well as the content of the page. Some of the search components I’ve seen have used multiple database queries to grab all the pages where the search query matched the title, another query to grab the pages that matched the keywords, another for the description, and finally one more for the content. Then moving those results into a bunch of arrays and applying some more logic to determine how relevant those results are and send them to the search results page. There isn’t anything particularly wrong with this, but I like to do things in the least number of database connections as possible, and to keep it as simple as possible from the code side of things.
Searching Based on Multiple Fields
Here’s the sql statement I’ve used – I’ll explain it below:
SELECT title, filename, sum(relevance)
FROM (
SELECT title, filename, 10 AS relevance FROM page WHERE title like ‘%about%’
UNION
SELECT title, filename, 7 AS relevance FROM page WHERE filename like ‘%about%’
UNION
SELECT title, filename, 5 AS relevance FROM page WHERE keywords like ‘%about%’
UNION
SELECT title, filename, 2 AS relevance FROM page WHERE description like ‘%about%’
) results
GROUP BY title, filename
ORDER BY relevance desc;
The sql is a little more complex than a basic select statement but if you’re familiar with unions then it’s pretty simple. All I’ve done is created a number of select statements that grab the pages where the field matches the search term. In this case, the search term is "about". I’ve searched through the title, filename, keywords, and description fields and left the content field off the search results. I didn’t want to include the content because I decided if the search isn’t found in the title, filename, keywords or description then it really isn’t very relevant for the search and even if it does have it in the content, it’s most likely a minor part of the content and not too helpful for the user.
I’ve manually included a relevance field in each of these select statements assigning a "point-value" to each page in those results in order to weight certain fields. These relevance points are totally up to you, and for the purposes of this example, I’ve given the results 10 points if the search word is in the title, 7 if it’s in the filename, 5 if it’s in the keywords, and 2 if it’s in the description.
Next, I union all of those search results together to create a large list of results. However, if a page appears in more than one of those select statements then we need to group them, so using a subquery, I’ve summed the relevance fields and then grouped by the filename and title then ordering the results descending by relevance.
Search Results
| title |
filename |
sum(relevance) |
| About the Chamber |
about-chamber |
17 |
| About our Programs |
programs |
10 |
| Policies |
policies |
2 |
| Opportunities |
opportunities |
2 |
Here the about page came up number one because it had the word about in the filename as well as in the title, and the programs page came in second since the word about was in the title, but not in the filename. There were some other pages that had the word in the description so they were only given 2 points. Obviously a search for the word about isn’t going to be very helpful, but to outline this example I think it worked well. Now I just need to simply output these results from mysql with php and the search results are done. I grabbed the title to output on the page and then filename to link to from the search results page.
Relevance Percentage
Rather than outputting the actual relevance score sometimes you’ll see a percentage of how relevant the page is. In our case that’s easy to do as well. Based on our formula of 10 points for the title, 7 for the filename, 5 for keywords, and 2 for description it’s possible to get a total of 24 points. To get the pages percentage relevance we just divide the relevance score from our query by 24 and then multiple by 100. For example, 17/24*100=70% so our about page was 70% relevant.
Tweaking for Better Search Results
You will need to tweak this formula as you test your search functionality, and adjust the weighting for each field to get the best results. You might also want to add another select statement that grabs all pages where the title is exactly the same as your search – in that case you might want to give them 100 points to ensure that that page is weighted much higher than the others. This will help you tweak your search results for the best user experience.
Full Text Search
I’ve provided the examples using the LIKE clause simply because most users will already be familiar with it, but also because the fields that I’m searching are relatively small – I wasn’t searching the entire contents field of the pages on the site. If I wanted to add the content field, I’d probably use full-text searching using the match() function, you can read up on that here.
3 Comments »
Posted May 8th, 2008 07:33:50 pm by Jarrod Goddard
Filed in Internet Marketing, SEO & PPC
Keyword research is one of a number of skills that any decent SEO or search marketer has to be good at in order to be successful. But are they the only ones who should be aware of keyword research tools like Keyword Discovery and Wordtracker?
Information architects, content writers, and usability professionals are just a few job titles, aside from SEOs, that need to be using tools like Wordtracker and Keyword Discovery to conduct a core component of their services.
Keyword Research for Usability
A big part of usability is making it easy for users to find what they’re looking for on your site, and navigate to the content they are trying to reach. As a usability expert you need to know how users think and how they’ll navigate and "use" your site. Keyword research is arguably the best way of finding out how users think when navigating the web. How do people search for your product or service? Do they commonly use industry acronyms and slang or is that left to industry professionals? As a usability consultant you’ll need to find this information when planning the links, content, and navigation of the site. Keyword research tools are among the best tools for this sort of research.
Keyword Research for Copywriting
A vital skill in copywriting is being able to communicate and speak to your user in language they use and understand. You need to be able to connect with them and make them think while they’re reading the content on your web site. If the copy on your site is going over their head, then it won’t work. Don’t use complicated or technical language that your users won’t recognize when you can keep it simple and write your content in their words. Once you work with a keyword research tool, you may find that a lot of people do search for your product or services using technically-advanced words and slang and that means you can use those types of words in your content, but most often you’ll find that you need to keep things simple and not try to impress your users with how well your content is written, but how well it speaks to them.
A lot of writers just like to write incorporating college-level vocabulary into their work and over-using adjectives that can really lose the attention of your audience on the web. The web is a unique medium not just because of how you navigate it, but also because of the attention span of a typical user. Most web users are inpatient and need to be sold to or communicated with quickly. Don’t drag out your message with content that doesn’t connect to your users immediately, in their language.
Keyword Research for Information Architects
How do you name the different sections of your site? Everyone has an "About Us" page on their site that provides background information on the company. Sometimes this page is called "About Us", "Company", "Profile" or "Firm". How do you choose which term to use? Do some keyword research.
In the web industry, I’ve found that people search for words like "web design company" much more than "web design firm" and rarely do people search for web development companies using the word "about" in their search phrase – so maybe it’s best to use the word "company" in your main navigation and refer to yourself as a "company" as opposed to a "firm" or "agency". Doing your keyword research will point this out to you and will help you make decisions on what keywords to use on your site and how to incorporate them into your navigation.
Bringing It All Together
As you start looking at this keyword research and incorporating language into your content, your internal links, your navigation, and overall site architecture you’ll find that a large part of your optimization is complete. Your SEO tasks will then consist of crafting title and META tags, creating internal links and doing some of the technical work like setting up redirects and rewriting, removing duplicate content, and ensuring that the site is crawlable – but much of the editing and fine tuning of content will be completed. Leaving less work for you as an SEO.
This also shows the advantages of having skills in each of these areas of web development. A lot of programmers have really bad usability skills, and a lot of writers, well, write a lot. Being a great SEO means having a strong knowledge of usability, writing, and content architecture and keyword research tools are a strong asset in every aspect of developing your web site strategy.
Comments »
Posted May 6th, 2008 03:41:46 pm by Jarrod Goddard
Filed in Linux, Web Hosting & Server Administration
A couple years ago I was doing some work on a client’s web site on a shared hosting environment at a small local ISP and I was setting up some file uploads with PHP and was having problems getting permissions setup and enabling them on the server. I had done it many times before but for some reason just couldn’t get it to work. I resorted to calling the ISP to ask if they had any non-standard setup or configuration on their server that would be causing me problems.
My Own Experiences With System Administrators
I was talking to their system administrator and was asking why I wasn’t able to do any file uploads. He took a few minutes and simply went to PHP.net and found their “handling file uploads in php” document and began reading it to me. I told him I had done it before and had never had any problems, but this was the first time I was trying to do so on their servers. He said to me “I’m just a system admin, I don’t know anything about PHP.” I was kind of shocked. I thought it was pretty important to have at least a basic knowledge of PHP if you’re going to be managing any number of Linux servers that run PHP.
This same thing has happened with another client’s site that has hosting elsewhere from my servers. Anytime their server is upgraded they recompile PHP and forget to enable the GD library. Then the back-end of the client’s site stops working and they contact me for help. After a simple test on the back end I can tell them that the GD library isn’t installed and they need to talk to their system administrator.
One other example was when I had to work on a large web site for a company that hosted their site internally. Their admins had been running Unix servers for years, but when I asked them if they could upgrade to PHP 5 from PHP4 they were really cautious about doing so. It’s not that they had many PHP applications on their server, but more that they were unsure how to do it and wanted to take a couple weeks to learn about what PHP was. That seemed a little odd to me.
Linux System Administrator Job Postings Not Even Requiring PHP
I even see a lot of job postings for Linux/BSD system admins and they don’t mention any requirement to have experience with PHP – sometimes I don’t even see PERL! Of course, it might be assumed to have that knowledge, but when you list things like experience with Sendmail, Apache, MySQL and SSH they should probably be including PHP and PERL in the mix as well.
Why Is It Important?
There probably isn’t much explanation necessary of why PHP should be required skill but when you need to setup PHP modules or be in charge of a server running PHP you need to have a basic understanding of the language. As a system admin, the server security is up to you, and if you aren’t aware of security exploits within PHP or at least know enough to secure your PHP installation – things like register_globals, allow_url_fopen, and magic_quotes – then you’re really not doing your proper research and shouldn’t be installing it on your server.
This goes for any module or program you install on your server. If you are in charge of the server you need to be knowledgeable of everything you put on it, and if you’re going to be setting up PHP, PERL, Sendmail, or an FTP program you need to be aware of their security implications. That’s your job as a system administrator, and it can be really frustrating having to deal with a system admin for a server you’re managing a web site on that can’t even talk to you about configuring the PHP installation because it’s over their head.
1 Comment »
