I actually stumbled across this question because I was looking if this has already been done. The only thing I found yet is the mycarmakesnoise app, but you'll have to compare sounds yourself. I am not a mechanic, so I don't know much about sounds of a defect in a car.

I am however starting to learn about deep learning. To me it seems totally possible to use the spectrogram of an audio recording of a defect to identity problems using a convolutional neural network. Maybe even better than humans can, as images can already be classified more accurate by a computer than a human (~94% for humans vs ~95% for computers). There are major issues to overcome though.

First of all, you need a very large data set. That means gathering (at least) hundreds of audio recordings of the same defect in the same vehicle. Thousands of recordings would be even better (bigger data set = better results). Let's say we want to identify ten different defects and there's (total guess) 300 car types. You would need 30010100=300.000 recordings for ten defects alone. Also, you can't use the same car over and over again. That would mean you'll need 100 cars of the same type per defect for best results. Gathering these recordings would be a massive operation. A way to counter this problem would be to start of with one brand and type of car, I guess.

The next problem is the difference in sound per car type with the same defect. A Volvo with busted brakes will sound different from a Ford with the same problem. This problem can probably be overcome by having a really big data set, as mentioned before.

Also, the background noise will always be different. One person might be sitting in a real silent car, listening to nothing but his thoughts. The other person will be blasting metal in a shit car with stuff creaking and stuff in the trunk making a lot of noise. The same thing counts here. You need a large data set, so the background noise can be as random as possible.

The last problem I could think of now is the computational power needed. Neural networks need to be trained using the big data set we collected. This takes a lot of computational power. Without access to a super computer or at least a real good videocard, training would take days up to weeks.

In short: Yes, it is probably possible to detect defects in cars by using the sound it makes, but it's probably really hard to do.

I actually stumbled across this question because I was looking if this has already been done. The only thing I found yet is the mycarmakesnoise app, but you'll have to compare sounds yourself. I am not a mechanic, so I don't know much about sounds of a defect in a car.

I am however starting to learn about deep learning. To me it seems totally possible to use the spectrogram of an audio recording of a defect to identity problems using a convolutional neural network. Maybe even better than humans can, as images can already be classified more accurate by a computer than a human (~94% for humans vs ~95% for computers). There are major issues to overcome though.

First of all, you need a very large data set. That means gathering (at least) hundreds of audio recordings of the same defect in the same vehicle. Thousands of recordings would be even better (bigger data set = better results). Let's say we want to identify ten different defects and there's (total guess) 300 car types. You would need 30010100=300.000 recordings for ten defects alone. Also, you can't use the same car over and over again. That would mean you'll need 100 cars of the same type per defect for best results. Gathering these recordings would be a massive operation. A way to counter this problem would be to start of with one brand and type of car, I guess.

The next problem is the difference in sound per car type with the same defect. A Volvo with busted brakes will sound different from a Ford with the same problem. This problem can probably be overcome by having a really big data set, as mentioned before.

Also, the background noise will always be different. One person might be sitting in a real silent car, listening to nothing but his thoughts. The other person will be blasting metal in a shit car with stuff creaking and stuff in the trunk making a lot of noise. The same thing counts here. You need a large data set, so the background noise can be as random as possible.

The last problem I could think of now is the computational power needed. Neural networks need to be trained using the big data set we collected. This takes a lot of computational power. Without access to a super computer or at least a real good videocard, training would take days up to weeks.

In short: Yes, it is probably possible to detect defects in cars by using the sound it makes, but it's probably really hard to do.

[edit] An option could be to train a neural network per car brand and model, but still. I can imagine for example a part breaking could produce very different sounds depending on where it broke. Again, I am no mechanic, so I might be full of shit when it comes to cars.

I don't think a generic problem-detection tool is next to impossible. I think it is very plausible a tool like this could be built using a neural network. Gathering enough sound data to train the network would be a hell of a job though.

Also, if we are working with newer cars we can add all kinds of information from the OBD system to the input tensors of the neural network. That extra information could be very helpful for the neural network to figure out what's going on.

I actually stumbled across this question because I was looking if this has already been done. The only thing I found yet is the mycarmakesnoise app, but you'll have to compare sounds yourself. I am not a mechanic, so I don't know much about sounds of a defect in a car.

I am however starting to learn about deep learning. To me it seems totally possible to use the spectrogram of an audio recording of a defect to identity problems using a convolutional neural network. Maybe even better than humans can, as images can already be classified more accurate by a computer than a human (~94% for humans vs ~95% for computers). There are major issues to overcome though.

First of all, you need a very large data set. That means gathering (at least) hundreds of audio recordings of the same defect in the same vehicle. Thousands of recordings would be even better (bigger data set = better results). Let's say we want to identify ten different defects and there's (total guess) 300 car types. You would need 30010100=300.000 recordings for ten defects alone. Also, you can't use the same car over and over again. That would mean you'll need 100 cars of the same type per defect for best results. Gathering these recordings would be a massive operation. A way to counter this problem would be to start of with one brand and type of car, I guess.

The next problem is the difference in sound per car type with the same defect. A Volvo with busted brakes will sound different from a Ford with the same problem.

The last problem I could think of now is the computational power needed. Neural networks need to be trained using the big data set we collected. This takes a lot of computational power. Without access to a super computer or at least a real good videocard, training would take days up to weeks.

In short: Yes, it is probably possible to detect defects in cars by using the sound it makes, but it's probably really hard to do.

I actually stumbled across this question because I was looking if this has already been done. The only thing I found yet is the mycarmakesnoise app, but you'll have to compare sounds yourself. I am not a mechanic, so I don't know much about sounds of a defect in a car.

I am however starting to learn about deep learning. To me it seems totally possible to use the spectrogram of an audio recording of a defect to identity problems using a convolutional neural network. Maybe even better than humans can, as images can already be classified more accurate by a computer than a human (~94% for humans vs ~95% for computers). There are major issues to overcome though.

First of all, you need a very large data set. That means gathering (at least) hundreds of audio recordings of the same defect in the same vehicle. Thousands of recordings would be even better (bigger data set = better results). Let's say we want to identify ten different defects and there's (total guess) 300 car types. You would need 30010100=300.000 recordings for ten defects alone. Also, you can't use the same car over and over again. That would mean you'll need 100 cars of the same type per defect for best results. Gathering these recordings would be a massive operation. A way to counter this problem would be to start of with one brand and type of car, I guess.

The next problem is the difference in sound per car type with the same defect. A Volvo with busted brakes will sound different from a Ford with the same problem. This problem can probably be overcome by having a really big data set, as mentioned before.

Also, the background noise will always be different. One person might be sitting in a real silent car, listening to nothing but his thoughts. The other person will be blasting metal in a shit car with stuff creaking and stuff in the trunk making a lot of noise. The same thing counts here. You need a large data set, so the background noise can be as random as possible.

The last problem I could think of now is the computational power needed. Neural networks need to be trained using the big data set we collected. This takes a lot of computational power. Without access to a super computer or at least a real good videocard, training would take days up to weeks.

In short: Yes, it is probably possible to detect defects in cars by using the sound it makes, but it's probably really hard to do.