Testing knives: objective results vs impressions

[Alarion]

Reading most knife magazines, both here in Italy and american ones, I've been disconcerted at the approximation reached by some knife "testing" methods I've seen.
We all know what the so called "scientific method" is: repatability of a process, direct connection between method and results.
I've seen all knds of tests performed on various knives, from splitting a nail in two, to piercing some steel sheet, to cutting various things, from carrots to free hanging manila rope, all followed by the writer's feelings on how a given knife performed.
Mind well: the writer's feelings are important, a sa knife is not something working alone, but a tool which will be heldd by a hand, so how the tool feels in the hand is important, but it's not necessarily an index of the tool performance.
Even more: it gives us no way to confront a given knife with another...

Which is the diameter of the split nail?
Which steel it's made of?
Does all the nails used, even if they are of the same diameter and steel come from the same production run?
How was the knife used to split the nail?
Without all these information, it's impossible to use any of the information gathered by putting the knife under such grievous abuse for anything useful. We just wasted time and spoiled an otherwise perfect knife.

Even a process apparently constant, such as buying 100 nails and using them to test two knives by splitting them putting the blade on the nail and then using a hammer to pound on the blade's spine are indeed extremely erratic and unpredictable.
The hammer could hit the blade at any angle, putting the edge under severe torque, over the obvious stress of cutting a nail.
The blade could very well chip or dent because of this, and we would be convinced that the blade is soft because the edge bent while cutting a simple nail, or too hard because it chipped.

Even cutting a 2" free hanging manila rope may seem a quite objective testing method. It isn't so. The way the edge is ground, the final sharpening grit and, more than anything else, the cutter's skill, will do an enormous difference, so even this method says pretty nothing. It just demonstrates that THAT specific knife, held by THAT specific guy, could cut THAT specific piece of manila rope. Hardly of any use at all. It doesn't say nothing about ay other knife, or how the average knife will perform in the hands of the average person or how it will cut any other material.

So.
What methods we could employ to obtain data from our testing which can be compared with those obtained by other people, testing other knives?

Well, first and foremost the methods must all be the same, otherwise we'll be comparing apples with oranges...

How can we devise testing methods that can satisfy these requirements?
Well, it's fairly simple: forces, materials, angles and such must be as constant as possible.
For example: we could test edge sturdyness by cutting the same wood nail, made of mild iron (so that steel alloy is no more a problem) by cerating a small die which will carry the nail and hold the blade perfectly vertical on it, and letting a constant, given weight fall from a given height on the blade, and seeing how many blows are needed to cut the nail completely, which damage the blade underwent, measuring its depth and width as we do in a Rockwell hardness test, or increase the weight till the blade chops the nail cleanly and noting the various damages done to the edge by the various weights.
This is just an example.
Yes, it requires building a little device.
Yes it can't be done in an impromptu way while playing with your new knife in the backyard, but it gives significant, repeatable, constant results that can serve to give an objective evaluation of the knife and that will allow, over time, to build a valuable database of knife performance.

 

[knifenerd]

Excellent post! I,too, have had similar concerns over the nature of knife "testing" that we see in the verious mags and here. It seems there is no really objective set of standards which the knife purchaser can use to evaluate knife performance.

Unfortunately, much of the info, although well-intentioned, is largely "anecdotal" in nature. Merely subjecting a blade to a sometimes absurd range of extremes has no meaning unless there are some known standards by which to judge performance. This is true with knives as well as any other situation where serious testing must take place.

Are such standards too difficult to promulgate due to the many differing types of blade steel (and now non-steels)? Or the many blade shapes and edge geometrys? Even differences in blade sizes?

Are such standards considered "taboo" by manufacturers or even custom makers? Would they be fearful of not being able to "make the cut"? (obvious horrible pun )

A good, active discussion of this area might be very useful. Feel free to jump in.

 

[Bobwhite]

I am all for standardized testing, maybe Cliff Stamp could put it together, I sure can't. We need something that follows the scientific method. It must be objective, measurable and repeatable.

 

[Cliff Stamp]

Alarion :

Which is the diameter of the split nail? Which steel it's made of? Does all the nails used, even if they are of the same diameter and steel come from the same production run? How was the knife used to split the nail?

It is true that the more information that is given about how the work was done the more you can learn from it, which is why you should be as detailed as possible when describing what you have done.

Even a process apparently constant, such as buying 100 nails and using them to test two knives by splitting them putting the blade on the nail and then using a hammer to pound on the blade's spine are indeed extremely erratic and unpredictable. The hammer could hit the blade at any angle, putting the edge under severe torque, over the obvious stress of cutting a nail. The blade could very well chip or dent because of this, and we would be convinced that the blade is soft because the edge bent while cutting a simple nail, or too hard because it chipped.

There are many variables in the above it is true. Nails are not very consistent and can see strengths changes, double even on the exact same type. As well, just as you noted, because people are not machines there will be a huge variation from swing to swing regarding speed, direction, follow through, etc. . However all of these problems are random, and can be handled by simply doing repeated work. It would be very poor method to cut through one nail with two knives and based on that make a judgment on how the blades held up. However if you selected 20 nails at random for each blade and then cut through all of them, and compared on the distribution of damage, that would be a robust result.

Even cutting a 2" free hanging manila rope may seem a quite objective testing method. It isn't so. The way the edge is ground, the final sharpening grit and, more than anything else, the cutter's skill, will do an enormous difference, so even this method says pretty nothing. It just demonstrates that THAT specific knife, held by THAT specific guy, could cut THAT specific piece of manila rope. Hardly of any use at all. It doesn't say nothing about ay other knife, or how the average knife will perform in the hands of the average person or how it will cut any other material.

Yes, skill plays a large part which is why you want to be using multiple knives at the very least always have a standard to compare against. If you have one person sharpen both knives and use them and he can consistently get one knife to cut through more strands of rope then it is reasonable to assume that knife will cut better on that rope for anyone who uses a similar technique. Different techniques will yield different results, and this can be explored as well. As for cutting other material, while there are differences in materials, you can make generalizations because they are also similarities. You just have to know how the performance correlates from one material to another.

Well, first and foremost the methods must all be the same, otherwise we'll be comparing apples with oranges...

While such work has its merits, consider that many times you don't want the same method used. For example if you were exploring the ability of a small parang to build a shelter as opposed to a hatchet, you would not want to use both in the exact same manner, because they perform at an optimal level with different techniques. It is interesting to use a hatchet as you would a parang, as you can learn from it, but it is unfair to judge from its performance the quality of the hatchet. You can learn from comparing towards an end goal, not on a specific technique. The main thing is how the work is interpreted, this has just as much part in it being science as how the data was collected.

For example: we could test edge sturdyness by cutting the same wood nail, made of mild iron (so that steel alloy is no more a problem) by cerating a small die which will carry the nail and hold the blade perfectly vertical on it, and letting a constant, given weight fall from a given height on the blade, and seeing how many blows are needed to cut the nail completely, which damage the blade underwent, measuring its depth and width as we do in a Rockwell hardness test, or increase the weight till the blade chops the nail cleanly and noting the various damages done to the edge by the various weights.

I do believe that standard testing would be of great benefit to the knife industry and to which I would really like to see the various makers and manufacturers get standard materials tests done on the steels they used with the heat treating methods they employ. It would be very informative if we had grain sizes, charpy values, tensile and yield strengths, ductility, wear and corrosion tests for all the materials at hand. I would also very much like to see work done as you describe with the nail cutting and it would add to the current body of information and it would give information on the cutting ability as well as hardness and toughness.

It is also very important to know how such material properties correlate to actual use. For example the biggest problem with nail contacts is not when you need to cut one in two, but when you hit one accidently, or something similar. This is a very dynamic impact and it is not well modeled by using a press to cut a nail in half. Even how the knife and nail (or whatever) is held through the impact has a large effect and both will be very vigorously held in place by a machine. For example I can take a common kitchen knife and using a vice press it through a 3.5" common nail. If I try to chop the nail in two the blade of the kitchen knife will get dented. To me, both types of work are useful.

There are three basic changes I would like to see. First off getting manufacturers and makers to have standard materials tests done on the steels they use after their heat treatment. Second, see knives compared to each other. Instead of saying "x" knife cuts well, compare it to the others you have on hand and say it cuts better than "y" but not as good as "z". Is this perfect? No. But is would be a huge step forward towards standard evaluations. Finally get makers and manufactures to commit to the actual intended purpose of a knife and how they feel it should be evaluated.

If the last one sounds odd, I have at times contacted manufactures and told them I was doing a review and asked for an outline of what the knife was intended to do and some example of work they felt would showcase its abilities. My intention is not to limit the review to just such work, in fact I would also pretty much do the exact opposite, however I would like to cover what it was designed to do. However at times manufacturers refuse to comment along those lines. Then the same people complain about the work that was done. Basically if you don't commit to how a knife should be judged you can always argue against the results.

To be clear I am not branding everyone with a failure on the above, I know makers and manufactures who do both of the above and reviewers who will be specific in talking about the relative performance of their knives. I would just like to see it more widespread. I would also like to see standard testing methods put forth, and in fact have made it clear since I began doing reviews that this was the case and I have received many interactions with makers and manufactures as well as ELU's on such.

I will make one rather important point, it is not exactly fair to criticize someone for not doing something when you are not willing to do it yourself. It is rather trivial to say do this and that, but I think that if you seriously want to see a change in how knives are being evaluated then you really should start some work yourself. Initiate some action. Do a review as you feel it should be done and put yourself up for the same type of judgment on method and interpretation.

-Cliff

 

[hd2k_va]

Yeah, that's all true. But how many knives have you bought because of extensive, scientific research and how many because they look good, feel good, and smell good? (Ok, forget about the smelling)

 

[mikemck]

Originally posted by hd2k_va
Yeah, that's all true. But how many knives have you bought because of extensive, scientific research and how many because they look good, feel good, and smell good? (Ok, forget about the smelling)

No, that is a good point indeed. I myself love the smell of a new Emerson knife, but I hate Emerson knives otherwise.
For me, a new Emerson knife actually smells better than any other new knife I have run across...seriously!

Some new knives have no smell at all of course...

 

[Samurai6]

Objective knife tests that are quanitfiable and repeatable are practically impossible. Knives are so individual that you are practically evaluating a knife's ability to be itself. Even if you could objectively measure knife performance, it probably wouldn't be very useful becuase the data wouldn't be meaningful to everyone.

To me, subjective tests are the only useful ones. I just want to know what kind of user the tester is, and how much he liked the knife. One way to make this sort of review interesting for others is to do two similar knives and write the review as a comparison and contrast between the two. Then the tester can say, I like this kind of knife for this reason, and knife A did this better than knife B. That way the reader can decide if he is the same kind of knife user and might also like knife A better than knife B.

Just to say knife A cut rope in my test machine at X kilograms of force is of no use to me. When I read Cliff's reviews, I always skip to the end to read his impressions and conclusions.

 

[Cliff Stamp]

Steve :

Even if you could objectively measure knife performance, it probably wouldn't be very useful becuase the data wouldn't be meaningful to everyone.

Yes, this is one of the major problems with doing controlled testing, the results tend to be complicated and numerical. However as long as you have multiple knives included this is not an impossible problem to solve. You can always use relative performance to gain meaning. For example If I say that an Opinel can cut through 3/8" hemp with 12.5 +/- 0.3 lbs does that mean anything to anyone? Probably not a lot of people because you would have to have done the same thing to interpret the result. However what if I say a Twistmaster from Cold Steel is 25 +/- 2 lbs, a SOG SEAL is 80 +/- 4 lbs etc. . If you have used any of these knives the numbers can all thus be ranked against the one you are familiar with and thus they gain meaning. The scaling is the critical part.

Knives are so individual that you are practically evaluating a knife's ability to be itself.

Well yes, they do cover a broad range of tools, as essentially a parang and a fillet blade are both knives. Essentially a review should be a showcase of what a knife can do well and what it does very poorly, compared to others in its class over a large range of work which covers the scope of indended use. Every knife will excell at something, and do poorly at something else. If you present the performance over a broad range of work then someone else can look at what was done and judge the suitability of the knife based on what they want it do to. A knife is in general neither "good" nor "bad", as this judgement is very dependent on the individual.

-Cliff

 

[Alarion]

Thanks Cliff, very interesting answer and, indeed, if the builders themselves would do some tests on their knife sit would save a lot of time.
The nail test was just an example: it's one of the most widely used, and one which I think is the least useful.
I agree with all you said. Obviously when saying that all knives should be equally tested I mean that all knives should be subjected to the tests relevant or suited for that knife type. Testing the lock strength of an integral knife would be quite pointless
As for your last statement, it is exactly what I intend to do and what I'm doing here in Italy.
I'll start devising some tests, non destructive ones, for the knives I own.
I occasionally write on a knife magazine, here, and I'm trying to devise testing methods that can lead to repeatable tests and objective results.

For Steve: In fact I said that the subjective feelings of the tester are very important, but they shouldn't be confuse for objective data, as often happens.
I'll make an example: The famous Luger of the German Army is a gun that put on a ransom rest shoots marvellously, but that is quite hard to shoot accurately as it comes.
Most 2" S&W snubbies will shoot in a dollar at 20 yards, if put on the same rest. Shoot it off hand and you are lucky if you can hit the target
Objective testing will lead to a very high accuracy rating, while practical use can lead to completely different results but, nonetheless knowing that the gun you own has a potential for extremely high accuracy is very useful!

I wrote this post both to "stir the waters" and to find some help in devising objective knife testing methods, ones that could be used by anybody, without complex machines, but building just small, simple, cheap devices, or that anybody agrees can lend to good, objective data.
As I have learned, there are methods that can seem objective at first sight that are instead very random.
Any help will be appreciated.

 

[Ed Caffrey]

Testing will(or should be) always a part of a maker's realm. However, testing is as individual as each person is. Each individuals who makes or uses a knife must decide what qualities he or she desires, and seek to produce or procure a knife with those desired qualities. We often want to place things into a large group and try to determine which is "best". Just not feesible with knives. An example would be placing a 440C blade againist an 52100 blade in a corrosion test............no brainer! But, the 52100 would easily out shine the 440C blade in toughness and ease of sharpening. The point is that there is NO "perfect" blade or blade material that will fit each person's needs or desires. That's one of the reasons that there are so many different materials, blade configurations, grinds, and methods out there. In the above posts I saw mention of conducting some tests on nails. What's the point? Any bladesmith can create a blade that will easily cut nails, but it will be done at the expense of durability (toughness).
I suppose the controversy will be forever onging as to standards for knife testing, but until everyone makes the same knife, from the same materials, with the same methods, there will never be a fair comparison. As I said earlier, it is up to the individual as to what qualities are desired, and up to each maker too. The real trick is to find a maker or company that holds the same values as you do, and get your hands on that knife!
Sorry, but I get a little wrapped around the axle when I see testing of knives that leans towards the "silly" side, like cutting nails and chopping up car fenders.

 

[knifenerd]

There was a similar post to this one in March 2000. It seemed to me to beat around the issues and no real conclusions were reached. I'm sorry I can't provide the link. I'll make it a New Year's Resolution to become more computer-literate.

I was glad to see Alarion raise the issue again. It would be very helpful if some manufacturers and/or makers would weigh into this topic. They are the ones with the most technical info on this entire issue. I still think it is possible to reach at least some objective norms that could be used to evaluate a knife's performance.

 

[Jerry Hossom]

Ed has it exactly right. What matters is really an issue of the person who will use the knife and their individual needs, likes, dislikes, and aesthetic interests. If this were math, all knives would fit into a common formula and look much the same.

I don't know many makers who don't do a good bit of testing. I correspond with several. Also most of us were knife users before we were knifemakers, and continue to be.

 

[Mike990]

Before buying I look at the sum total of objective and subjective comments of as many people as I can find. I then take a shot at it and buy the knife or decline. Of course this is only for knives that interested me in the first place and that I could afford at the time. When I read stuff over time here I remember it. I also do searches and e-mail owners. Testing is the sum total of all this. I have a good idea of what to expect anyway. In the world of knives not much is magic. Differences are style, quality of materials, execution and design. A certain thickness and style grind of ATS-34 heat treated properly to a certain RC usually performs in a way that is to be expected in cutting performance. That's when the other factors start to take over. Everything is also a compromise to one degree or another. You give this and you get that. I don't think any type of test would ever satisfy everyone.

 

[bandaidman]

The overall lack of objective, independent, and scientific testing in the cutlery realm is disappointing.

Some major companies like Buck and Spyderco have made significant investments in testing equipment and their contributions to the industry are to be commended. I recall a recent thread where Sal Glesser offered to test some of Rob Simonich's steel and that generosity speaks volumes about the quality of the man and his corporation.

What is lacking though is a "consumer reports" type testing between the different steels and companies.

Many custom makers test their blades but can not afford to destroy too many blades as that would be prohibitably expensive. Some individuals try to test blades but the results are quite mixed.

This is an area where the major knife magazines have really fallen short. I would love to see them test a large sample of various factory blades in a scientific manner and publish the results. Alas, the threat of the loss of advertising dollars seems to preclude that.

We are thus left with subjective reports and opinions

 

[Rob Simonich]

Many custom makers test their blades but can not afford to destroy too many blades as that would be prohibitably expensive.

Exactly right in my case. I would love to do a lot more destructive testing in my shop. I do as much testing both non destructive as well as destructive that time permits, especially when wringing out a new steel or different heat treat. I never farm out my destructive testing, I cant learn near as much from an e-mail or a phone call on how a blade behaved when it gave up the ghost. On the other hand, I rely heavily on customer feedback with field testing. I can compare their notes about a blade with mine.

Some individuals try to test blades but the results are quite mixed.

This is why I think the data collected from machines like Spyderco's CATRA are important. They are repeatable.

Lots of good info in this thread, just way to many big words for me! Gives me a headache!

 

[Jerry Hossom]

Rob, it's not the words. It's the Guinness...

 

[Alarion]

Originally posted by Ed Caffrey
Testing will(or should be) always a part of a maker's realm. However, testing is as individual as each person is. Each individuals who makes or uses a knife must decide what qualities he or she desires, and seek to produce or procure a knife with those desired qualities. We often want to place things into a large group and try to determine which is "best". Just not feesible with knives. An example would be placing a 440C blade againist an 52100 blade in a corrosion test............no brainer! But, the 52100 would easily out shine the 440C blade in toughness and ease of sharpening. The point is that there is NO "perfect" blade or blade material ...

I know all this and all the other issues raised by the others here above.
I know perfectly that many of the features that make us buy a knife are very subjective, and that style, alone, plays a great role.

Let be more clear with an example, speaking of something that everyone of us knows very well, well as knives or even more, and that raises quite often the same passions and has a very wide set of features which may be interesting (as a knife): cars.

Unlike knives, cars gets tested in a repeatable manner and results are published on magazines with objective reports regarding performance, quality and also subjective impressions.
The average quality car magazine will provide acceleration, fuel consumption, maximum speed and engine efficiency data, along with quality of fittings compared with models in the same category and price range.

Obviously knowing that a given car goes from 0 to 100 Kmh in 10 seconds is not everything about it, nor the fact that it can go at 195 Kmh, but these are _useful_ data to know, and if they aren't the only ones upon which our choice of a car will rely upon, they are nonetheless very significant, and given the same basic characteristics (number of seats, sedan or station wagon etc), the same price and the same quality, I think that nobody of those writing here would choose a car with 18 seconds 0-100Kmh and 130 Kmh max speed over one with 9 seconds 0-100 and 200 max speed

What I mean is exactly this: we don't know the "speed" of our knives, nor the fuel consumption, nor the acceleration... We choose basing our choice on a wide sum of subjective impressions that add up to a final judgement.
Yes, it works, but it's not the best way to proceed and does nothing to make things easier for the novice or to set minimum standards.

I think that knives should be tested and compared against other knives in the same class, and that tests are done for cutting ability, edge retention, corrosion resistance, lock quality and resistance under static and dynamic forces.
Moreover efficiency for tasks directly related to the knife, quality of fittings, precision of manufacture should also be evaluated as not quantifiable but comparable features, against those of knives took as "standard" (for example a production knife coulde be compared against a high-quality custom and two ther famous firms knives).

Yes, testing a 52100 knife for JUST corrosion resistance against one made of 440A would be very unfair.
But I never said it should be done this way!
EACH characteristic should be tested and, once you have objective data wherever possible and an evaluation against various other knives in the fields where objective comparation is not possible, one will choose.

If I don't give a damn for edge retention as I know I will use the knife just for cutting bread and putting chocolate cream on it, and I don't want bloody protective oil on my bread together with the choco cream, I'll chose the 440A knife, even if it won't retain a decent edge for much time even going thru choco cream.
If I have to skin deers, then I'll go for the 52100 blade.

Objective testing of a knife means also you must NOT presume to know what use the knife will be put thru, but just taking measurements and putting them in a useful format so that they are available to potential customers. Let the customer draw its own conclusions.
Subjective issues as ease of use and efficiency will obviously have to keep the use of the knife in account, but that is another matter entirely.

 

[Alarion]

This is how to test a blade for rust resistance alone.

Take a non galvanized mild steel washer.
wipe both the washer and the blade perfectly clean with acetone.
Dip in distilled water with 1% in weight salt in it (10 grams of salt in 1 litre).
Put on a shelf and wait till the first rust spots appear, measure time between the first rust spots on blade or washer and the first rust spots on the other object.
It may take hours, or days.
If after 24 hours still there isn't rust, dip again in the same water and wait.
Repeat till the first rust spots appear.
Take not of both the time the blade needed to show the first rust spots and the time bewteen balde and washer rusting, noting also which one rusted first.

This test is not destructive and if the blade is cleaned and oiled as soon as the first rust spots appear, the blade will not be subject to any damage.

Rust resistance will depend from the steel and its finish.

Is this test objective enough and easily repeatable?
Does anybody think there is a better/easier/more efficient method?

 

[knifenerd]

A round of applause to the contributors to this thread. Some of the most thoughtful and well-reasoned opinions I've seen recently have been expressed here.

The issue is perhaps more intricate than I had originally thought. I can see the vast number of problems that one would encounter in trying to formulate objective standards. I still think that some basic parameters could be devised.

Alarion's comparison to the auto industry is most appropriate, IMO. Cars are catagorized in different classes, i.e. sports sedans, SUVs, full-sized sedans,etc. No effort is wasted in trying to compare the cornering ablilty of an SUV to a sports car. Knives can similarly be broken down into various groups.

I'd still like to see some manufacturers chime in on the topic.

Thanks for starting a very thought-provoking thread, Alarion.

 

[Cliff Stamp]

Alarion :

'll start devising some tests, non destructive ones, for the knives I own.

I'd appreciate being kept up to date on what you come up with. I also have a couple of knives you could use as benchmarks. First off, you might want to outline the areas that you are going to evaluate, the aspects of knife performance that are going to be covered. Then you can work to develop testing schemes to rank blades in those areas. I have spent a fair amount of time thinking about this, in some fields I am pretty close to a decent testing method, in others pretty far away as they are quite complicated, handle security for example.

Considering the corrosion test, I have found that in regards to knives it is not just which steel can resist rust the longest that is important, which the above would quantify well, but also the type of rusting induced. For example I have found that most of the pseudo-stainless steels like ATS-34 will pit very badly in prolonged exposure to salt water. Even though they resist surface corrosion better than say 1095 or INFI. On long soaks of both I have seen less damage on the tool steels. One of the things you might want to look at is the reduction in cutting ability, edge retention, and edge durability with prolonged exposure to salt water. Also ease of rust removal, which is dependent on the type and depth of corrosion.

-Cliff