how to remove unwanted signal

I have a signal that is roughly in the shape of a sine wave rectifier. At random intervals there are square wave pulses inserted in the signal. I wish to automatically remove the square wave signal whenever it occurs.

I could clean up the signal by interpolating in between the gap where the square pulse exists. I am struggling to do this automatically could you please give me suggestions on how to detect the pulse edge of the signal so that i can determine the points to interpolate.

Answer 1

[Youssef Khmou]

hi,

One you know the mathematical formula of the square pulse you subtract it from your signal, or transform you signal into frequency domain and truncate the frequency of the square pulse,

Can you post an example of your data?

Answer 2

[rafael gonzalez]

Here is a copy of the signal for 60 seconds.

15716 16058 15830 15950 15850 16102 15942 15920 16244 16205 16193 15888 15933 16387 16274 16246 16116 16491 16392 16388 16238 16273 16549 16169 16376 16307 16478 16417 16297 16487 16582 16431 16817 16408 16862 16506 16437 16477 16739 16699 16644 16802 16907 16783 16965 16848 16817 16862 16787 17026 16678 16882 16901 16960 17057 17095 17125 17261 17214 17197 16915 16981 17150 17217 17378 17189 17243 17373 17149 17160 17379 17400 17559 17237 17193 17179 17191 17428 17243 17660 17444 17560 17594 17698 17637 17318 17754 17661 17670 17356 17533 17692 17612 17758 17609 17879 17789 17589 17829 18015 18015 18026 17689 17995 18006 17845 18075 18106 18129 17668 17866 18181 17882 18159 17876 17955 17969 18230 18024 18139 18198 18068 18142 17815 18113 18079 18093 18004 18242 18040 18335 18237 18055 18390 18160 18027 18314 18147 18403 18361 18226 18101 18121 18054 18179 18077 18219 18148 18477 18477 18438 18375 18149 18508 18101 18343 18166 18197 18371 18299 18182 18162 18373 18192 18377 18560 18407 18677 18580 18398 18472 18637 18532 18701 18628 18549 18695 18332 18700 18604 18220 18348 18687 18305 18277 18528 18425 18336 18575 18329 18366 18527 18371 18311 18483 18384 18569 18291 18717 18636 18668 18558 18475 18425 18577 18375 18617 18444 18528 18616 18260 18454 18345 18611 18620 18474 18588 18196 18484 18236 18627 18650 18162 18548 18307 18548 18590 18632 18245 18333 18500 18238 18212 18291 18504 18087 18433 18574 18504 18158 18093 18327 18321 18208 18482 18448 18243 18484 18345 18320 18061 18320 18152 18273 17995 18310 18289 18123 18000 18076 18080 17850 17841 17905 18165 17998 18033 17816 18202 18130 17834 18204 18091 18117 18138 17698 17772 18039 17718 18034 17610 17864 17870 17668 18037 17816 17746 17856 17873 17538 17465 17753 17745 17518 17497 17639 17700 17788 17468 17640 17497 17388 17336 17638 17446 17307 17382 17618 17446 17525 17362 17427 17248 17561 17490 17256 17135 17348 17447 17318 17413 17002 17121 16984 16936 16943 17346 17014 17309 16954 17110 16903 16988 17168 16865 17153 16911 16989 16651 16949 16698 17006 16617 16540 16858 16816 16914 16860 16902 16896 16772 16404 16514 16423 16481 16471 16407 16722 16389 16211 16214 16557 16641 16329 16337 16189 16426 16500 16490 16407 16437 16047 16291 16029 16108 16370 16321 16278 16026 16084 16238 15928 16061 15938 16093 15927 16026 15795 16025 16030 16052 16262 16275 16141 16011 16370 16204 16203 16204 16098 16059 16201 16287 16365 16254 16474 16450 16336 16234 16265 16675 16473 16730 16579 16487 16524 16565 16660 16808 16464 16621 16875 16531 16780 16969 16802 16784 16838 16673 16652 16713 17023 16734 17028 16858 17177 17087 16891 17057 17041 16903 17120 17192 16893 17358 17152 17010 17392 17038 17237 17153 17322 17196 17434 17091 17540 17218 17601 17264 17441 17530 17255 17519 17401 17515 17491 17595 17641 17428 17640 17413 17672 17535 17641 17777 17631 17807 17750 17678 17892 17576 17812 17594 17100 16694 16752 16472 16655 16502 16498 16353 16835 16626 16724 16819 16464 16596 16831 16663 16561 16746 16715 16850 16644 16719 16928 17024 16740 16716 16809 17023 16783 16773 17573 18039 18055 18165 18434 18165 17955 18149 18334 18483 18105 17987 18332 18079 18128 18086 18072 18275 18207 18105 18217 18419 18092 18439 18383 18535 18252 18378 18353 18327 18544 18620 18589 18294 18275 18558 18371 18657 18206 18610 18430 18239 18556 18503 18669 18275 18212 18701 18334 18345 18488 18466 18533 18685 18564 18258 18469 18654 18359 18409 18698 18389 18522 18685 18637 18576 18351 18417 18563 18675 18343 18707 18476 18320 18381 18660 18372 18626 18258 18495 18521 18321 18264 18388 18612 18453 18273 18687 18472 18220 18201 18598 18202 18246 18305 18456 18378 18487 18553 18583 18149 18339 18120 18268 18589 18494 18390 18210 18310 18194 18105 18077 18484 18268 18499 18208 18091 18438 18305 18151 18102 18243 17992 17936 17976 18330 18296 17911 18032 18044 17290 16580 17012 16649 16617 16645 16795 16636 16920 16683 16490 16469 16922 16546 16497 16875 16647 16498 16836 16685 16497 16657 16528 16803 16498 16742 16265 16402 16676 16551 16853 17675 17533 17657 17538 17874 17607 17556 17489 17697 17769 17722 17543 17695 17536 17374 17721 17324 17693 17487 17212 17543 17443 17554 17223 17511 17437 17383 17424 17043 17350 17212 16984 17194 17057 17352 16979 17220 17280 16817 17096 16786 16898 16808 16842 17052 16741 17152 16764 17086 16715 16726 16969 16808 16602 16796 16722 16714 16682 16640 16547 16799 16640 16373 16788 16308 16649 16343 16631 16706 16513 16672 16342 16606 16243 16106 16382 16533 16533 16327 16149 16064 16317 15936 16348 16063 16186 16253 15858 16286 16086 15917 15866 15933 16070 15906 15957 15691 15826 16238 16119 15889 16276 16079 16258 16268 16192 15954 16305 16001 16235 15999 16452 16061 16336 16370 16136 16116 16580 16202 16430 16206 16612 16478 16419 16329 16694 16421 16606 16589 16546 16514 16872 16705 16632 16771 16841 17015 17009 16629 17019 16678 16859 16962 16717 16894 17018 17173 16880 17090 16774 17138 17171 17060 17062 16981 16958 17139 17163 17217 17287 17006 16409 16119 16238 15787 16013 16069 15808 16064 16027 16315 16384 16015 16218 16099 16268 16408 16173 16413 16051 16353 16163 16154 16566 16418 16378 16542 16247 16296 16514 16244 16907 17666 17775 17757 17799 17700 17583 17731 17719 17628 17985 17659 17647 18034 18080 17945 17848 18079 18176 17840 18241 18029 18210 18179 18252 18181 18182 18077 18285 18154 18042 17919 18285 18157 18325 18030 17937 17938 17951 18119 18129 18140 18228 18236 18406 18202 18108 18095 18294 18121 18525 18065 18263 18205 18280 18169 18407 18472 18252 18398 18329 18518 18487 18237 18397 18477 18232 18166 18626 18298 18417 18479 18443 18529 18599 18698 18240 18292 18203 18577 18225 18274 18697 18330 18590 18240 18491 18543 18547 18427 18346 18431 18259 18463 18249 18592 18651 18522 18387 18654 18460 18421 18573 18548 18491 18343 18462 18446 18505 18434 18563 18358 18237 18385 18209 18623 18312 18369 18225 18666 18613 18317 18262 18364 18642 18456 18604 18354 18464 18603 18480 18464 18575 18621 18109 18346 18325 18577 18564 18456 18459 18402 18448 18241 18115 18197 18475 18442 18030 18469 17990 18184 18250 18312 18147 18413 17941 18091 18341 18211 17988 17896 18038 18143 18006 18338 17834 18181 18217 18060 17938 18185 17839 18125 17725 17767 17863 17870 18165 17832 17796 18056 17870 18065 17843 17966 17985 17789 17680 17633 17889 17588 17599 17543 17453 17746 17818 17683 17621 17614 17630 17586 17516 17632 17374 17689 17384 17276 17710 17460 17329 17380 17620 17321 17522 17495 17480 17245 17073 17280 17281 17505 17252 17225 17373 17032 16950 17359 16971 16922 16916 17005 17277 16810 17027 16822 16726 16820 16958 16879 17068 16921 16882 16665 16960 16607 16697 16976 16951 16528 16608 16666 16643 16734 16523 16601 16576 16781 15919 15293 15170 15196 15005 15203 15262 15037 15272 15042 15267 15146 14789 15170 14804 14821 14742 15044 14984 15130 14983 14981 14983 14808 14786 14985 14859 14766 14603 14707 15054 16109 15809 16158 15813 16090 16115 16229 15870 15926 15894 16143 16396 16043 16037 16251 16300 16478 16068 16254 16250 16183 16537 16334 16633 16609 16342 16502 16739 16486 16321 16698 16337 16456 16519 16818 16427 16867 16924 16622 16748 16560 16805 16761 16619 17094 16781 16755 16698 17037 16684 16828 16888 16761 16966 16854 17245 17328 17210 17261 17135 17401 16940 17200 17430 17050 17429 17423 17370 17426 17325 17489 17588 17423 17314 17313 17320 17465 17327 17347 17300 17606 17414 17585 17379 17499 17639 17419 17801 17801 17794 17582 17818 17678 17476 17944 17477 17779 17807 17552 17807 17657 17613 17911 18018 18054 17742 17834 17849 18052 18056 17888 17981 18020 18214 18114 17897 17845 18120 18281 18033 18285 18330 18287 17856 18030 18355 18157 18265 17906 18298 18353 18407 18116 18168 18281 18396 18398 18240 18384 18025 18354 18406 18521 18314 18522 18354 18544 18060 18433 18578 18426 18412 18419 18367 18154 18341 18292 18540 18380 18189 18657 17679 17140 16975 17203 17152 16966 16990 17023 16996 17249 17384 17261 17138 17359 17331 16942 17204 17234 17202 17226 16959 17394 17312 16973 17042 17291 16984 17172 17357 17128 17776 18571 18259 18433 18457 18592 18448 18246 18433 18536 18694 18376 18477 18419 18223 18432 18369 18703 18571 18203 18701 18489 18484 18579 18447 18250 18442 18369 18639 18655 18306 18315 18377 18154 18181 18592 18538 18436 18356 18501 18361 18227 18117 18067 18208 18475 18052 18385 18340 18358 18419 18338 18226 18455 18210 17985 18078 18260 18295 17961 18384 18173 18340 17955 18081 18324 18089 18085 17987 18162 17872 17961 18260 17945 18090 17767 18042 17797 18220 17713 17951 18054 18180 18124 18074 18115 17951 18052 17804 18029 17800 17784 17925 17982 17553 17977 17864 17834 17796 17681 17801 17560 17834 17414 17527 17643 17418 17668 17614 17698 17592 17468 17362 17525 17323 17309 17160 17334 17301 17126 17131 17200 17222 17269 17137 17148 17434 17251 17225 17401 16945 17141 16877 17089 17107 16898 16813 17173 16799 16957 17037 17063 17182 16921 16956 16920 16701 16699 16940 16650 16977 16938 16818 16863 16661 16906 16885 16440 16462 16480 16395 16601 16559 16765 16651 16332 16710 16217 16467 16279 16245 16393 16381 16480 16548 16140 16297 16278 15959 16090 16249 16236 16235 15882 16290 15935 16043 16175 15929 15746 15785 16130 15820 15981 16027 15971 16103 16261 15934 16264 15841 16008 16017 16184 16175 16195 16432 16097 16350 16243 16357 16093 16260 16257 16362 16172 16186 16329 16719 16728 16725 16306 16331 16679 16455 16657 16663 16686 16532 16622 16778 16578 16870 16958 16765 16995 16699 16790 16684 16941 17061 17019 16975 16753 16875 17223 17252 17141 17173 16954 16847 17092 17383 17141 17135 17300 17237 17020 17118 17459 17469 17260 17277 17181 17387 17496 17487 17644 17478 17378 17301 17220 17357 17601 17294 17291 17747 17650 17384 17550 17853 17563 17551 17534 17813 17851 17870 17606 17611 17666 17628 17545 18013 17742 17659 17862 17988 17869 17925 17842 17794 18016 18083 18021 18079 18015 17720 18003 17955 18214 17806 17844 17898 18091 17970 18036 18051 18309 17864 18049 18269 18053 18255 18286 18256 18303 18243 18176 18449 18219 18164 18358 18437 18397 18401 18130 18221 18181 18491 18180 18083 18193 18105 18235 18228 18368 18106 18603 18609 18542 18597 18333 18571 18566 18381 18469 18246 18300 18403 18478 18253 18581 18639 18436 18609 18367 18233 18681 18547 18435 18644 18685 18680 18569 18475 18704 18432 18638 18309 18235 18412 18733 18546 18443 18616 18457 18248 18455 18513 18277 18636 18374 18649 18376 18585 18379 18506 18445 18681 18516 18488 18492 18600 18346 18192 18418 18686 18484 18323 18528 18249 18540 18411 18190 18340 18653 18207 18423 18453 18634 18516 18419 18500 18273 18527 18468 18104 18519 18288 18276 18358 18306 18217 18483 18096 18081 18122 18003 18085 18127 18450 18079 17943 18057 17975 17925 18189 17973 17883 18028 17875 17872 18193 18036 18163 17837 18032 18076 18163 18184 18079 18175 18034 17979 18127 17922 17736 17806 17877 17744 17657 17954 18035 17747 17762 17933 17982 17558 17876 17763 17890 17767 17810 17648 17447 17626 17882 17530 17527 17344 17502 17758 17449 17583 17539 17748 17283 17254 17443 17461 17338 17186 17449 17538 17555 17370 17391 17277 17210 17383 17276 17087 17145 17001 17323 17400 17174 17327 16975 17067 17083 16835 17269 17140 16961 17129 16793 17078 17034 16817 16969 17014 16867 17059 16564 16738 16669 16681 16865 16773 16459 16813 16869 16842 16331 16534 16339 16591 16681 16398 16394 16448 16468 16486 16383 16249 16105 16561 16320 16229 16364 16333 16413 15939 16265 16181 16258 16207 16289 15824 16046 16093 15973 15930 16191 16137

Thanks for taking the time to look at the problem.

Answer 3

[Image Analyst]

Do you have the Signal Processing Toolbox? You could use diff() to identify drops greater than a certain amount and then use a Savitzky-Golay filter to smooth the signal by fitting it to a local polynomial. Or you could try to do an fft on the signal and do a really low pass filter on the signal by zeroing out any frequencies greater than the frequency of the sine wave.

Answer 4

[rafael gonzalez]

I managed to solve my problem by just recreating a signal with the similar shape and after filtering the original signal i applied a scale and offset to the new signal i created.

max(original_signal) - min(original_signal) + min(original_signal);

[Image Analyst]

max(original_signal) - min(original_signal) + min(original_signal);

does not look right because the last two terms cancel out. You didn't tell us that we were able to synthesize a "perfect" signal - I thought all we had to work with was the original signal.

[rafael gonzalez]

sorry the correct term is

average = (max(original) +  min(original))/2;

%amplitude of 1 * scale and offset

half_of_sinewave * (average) + min(original);

it's still not ideal because it only works if the timing is perfect. i need something to sync the two signals together if they are out of sync.