Html Question posted in
Some basic documentation can be found here.

Python get request produces different HTML than view source

import requests
# Request to website and download HTML contents
url='https://beacon.schneidercorp.com/Application.aspx?AppID=165&LayerID=2145&PageTypeID=2&PageID=1104&KeyValue=0527427230'
req=requests.get(url, 'html.parser')

produces this result:

'<!DOCTYPE html><html lang="en-US"><head><title>Just a moment...</title><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta http-equiv="X-UA-Compatible" content="IE=Edge"><meta name="robots" content="noindex,nofollow"><meta name="viewport" content="width=device-width,initial-scale=1"><style>*{box-sizing:border-box;margin:0;padding:0}html{line-height:1.15;-webkit-text-size-adjust:100%;color:#313131}button,html{font-family:system-ui,-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Helvetica Neue,Arial,Noto Sans,sans-serif,Apple Color Emoji,Segoe UI Emoji,Segoe UI Symbol,Noto Color Emoji}@media (prefers-color-scheme:dark){body{background-color:#222;color:#d9d9d9}body a{color:#fff}body a:hover{color:#ee730a;text-decoration:underline}body .lds-ring div{border-color:#999 transparent transparent}body .font-red{color:#b20f03}body .pow-button{background-color:#4693ff;color:#1d1d1d}body #challenge-success-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSIgdmlld0JveD0iMCAwIDI2IDI2Ij48cGF0aCBmaWxsPSIjZDlkOWQ5IiBkPSJNMTMgMGExMyAxMyAwIDEgMCAwIDI2IDEzIDEzIDAgMCAwIDAtMjZtMCAyNGExMSAxMSAwIDEgMSAwLTIyIDExIDExIDAgMCAxIDAgMjIiLz48cGF0aCBmaWxsPSIjZDlkOWQ5IiBkPSJtMTAuOTU1IDE2LjA1NS0zLjk1LTQuMTI1LTEuNDQ1IDEuMzg1IDUuMzcgNS42MSA5LjQ5NS05LjYtMS40Mi0xLjQwNXoiLz48L3N2Zz4=)}body #challenge-error-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSI+PHBhdGggZmlsbD0iI0IyMEYwMyIgZD0iTTE2IDNhMTMgMTMgMCAxIDAgMTMgMTNBMTMuMDE1IDEzLjAxNSAwIDAgMCAxNiAzbTAgMjRhMTEgMTEgMCAxIDEgMTEtMTEgMTEuMDEgMTEuMDEgMCAwIDEtMTEgMTEiLz48cGF0aCBmaWxsPSIjQjIwRjAzIiBkPSJNMTcuMDM4IDE4LjYxNUgxNC44N0wxNC41NjMgOS41aDIuNzgzem0tMS4wODQgMS40MjdxLjY2IDAgMS4wNTcuMzg4LjQwNy4zODkuNDA3Ljk5NCAwIC41OTYtLjQwNy45ODQtLjM5Ny4zOS0xLjA1Ny4zODktLjY1IDAtMS4wNTYtLjM4OS0uMzk4LS4zODktLjM5OC0uOTg0IDAtLjU5Ny4zOTgtLjk4NS40MDYtLjM5NyAxLjA1Ni0uMzk3Ii8+PC9zdmc+)}}body{display:flex;flex-direction:column;min-height:100vh}body.no-js .loading-spinner{visibility:hidden}body.no-js .challenge-running{display:none}body.dark{background-color:#222;color:#d9d9d9}body.dark a{color:#fff}body.dark a:hover{color:#ee730a;text-decoration:underline}body.dark .lds-ring div{border-color:#999 transparent transparent}body.dark .font-red{color:#b20f03}body.dark .pow-button{background-color:#4693ff;color:#1d1d1d}body.dark #challenge-success-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSIgdmlld0JveD0iMCAwIDI2IDI2Ij48cGF0aCBmaWxsPSIjZDlkOWQ5IiBkPSJNMTMgMGExMyAxMyAwIDEgMCAwIDI2IDEzIDEzIDAgMCAwIDAtMjZtMCAyNGExMSAxMSAwIDEgMSAwLTIyIDExIDExIDAgMCAxIDAgMjIiLz48cGF0aCBmaWxsPSIjZDlkOWQ5IiBkPSJtMTAuOTU1IDE2LjA1NS0zLjk1LTQuMTI1LTEuNDQ1IDEuMzg1IDUuMzcgNS42MSA5LjQ5NS05LjYtMS40Mi0xLjQwNXoiLz48L3N2Zz4=)}body.dark #challenge-error-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSI+PHBhdGggZmlsbD0iI0IyMEYwMyIgZD0iTTE2IDNhMTMgMTMgMCAxIDAgMTMgMTNBMTMuMDE1IDEzLjAxNSAwIDAgMCAxNiAzbTAgMjRhMTEgMTEgMCAxIDEgMTEtMTEgMTEuMDEgMTEuMDEgMCAwIDEtMTEgMTEiLz48cGF0aCBmaWxsPSIjQjIwRjAzIiBkPSJNMTcuMDM4IDE4LjYxNUgxNC44N0wxNC41NjMgOS41aDIuNzgzem0tMS4wODQgMS40MjdxLjY2IDAgMS4wNTcuMzg4LjQwNy4zODkuNDA3Ljk5NCAwIC41OTYtLjQwNy45ODQtLjM5Ny4zOS0xLjA1Ny4zODktLjY1IDAtMS4wNTYtLjM4OS0uMzk4LS4zODktLjM5OC0uOTg0IDAtLjU5Ny4zOTgtLjk4NS40MDYtLjM5NyAxLjA1Ni0uMzk3Ii8+PC9zdmc+)}body.light{background-color:transparent;color:#313131}body.light a{color:#0051c3}body.light a:hover{color:#ee730a;text-decoration:underline}body.light .lds-ring div{border-color:#595959 transparent transparent}body.light .font-red{color:#fc574a}body.light .pow-button{background-color:#003681;border-color:#003681;color:#fff}body.light #challenge-success-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSIgdmlld0JveD0iMCAwIDI2IDI2Ij48cGF0aCBmaWxsPSIjMzEzMTMxIiBkPSJNMTMgMGExMyAxMyAwIDEgMCAwIDI2IDEzIDEzIDAgMCAwIDAtMjZtMCAyNGExMSAxMSAwIDEgMSAwLTIyIDExIDExIDAgMCAxIDAgMjIiLz48cGF0aCBmaWxsPSIjMzEzMTMxIiBkPSJtMTAuOTU1IDE2LjA1NS0zLjk1LTQuMTI1LTEuNDQ1IDEuMzg1IDUuMzcgNS42MSA5LjQ5NS05LjYtMS40Mi0xLjQwNXoiLz48L3N2Zz4=)}body.light #challenge-error-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSI+PHBhdGggZmlsbD0iI2ZjNTc0YSIgZD0iTTE2IDNhMTMgMTMgMCAxIDAgMTMgMTNBMTMuMDE1IDEzLjAxNSAwIDAgMCAxNiAzbTAgMjRhMTEgMTEgMCAxIDEgMTEtMTEgMTEuMDEgMTEuMDEgMCAwIDEtMTEgMTEiLz48cGF0aCBmaWxsPSIjZmM1NzRhIiBkPSJNMTcuMDM4IDE4LjYxNUgxNC44N0wxNC41NjMgOS41aDIuNzgzem0tMS4wODQgMS40MjdxLjY2IDAgMS4wNTcuMzg4LjQwNy4zODkuNDA3Ljk5NCAwIC41OTYtLjQwNy45ODQtLjM5Ny4zOS0xLjA1Ny4zODktLjY1IDAtMS4wNTYtLjM4OS0uMzk4LS4zODktLjM5OC0uOTg0IDAtLjU5Ny4zOTgtLjk4NS40MDYtLjM5NyAxLjA1Ni0uMzk3Ii8+PC9zdmc+)}a{background-color:transparent;color:#0051c3;text-decoration:none;transition:color .15s ease}a:hover{color:#ee730a;text-decoration:underline}.main-content{margin:8rem auto;max-width:60rem;width:100%}.heading-favicon{height:2rem;margin-right:.5rem;width:2rem}@media (width <= 720px){.main-content{margin-top:4rem}.heading-favicon{height:1.5rem;width:1.5rem}}.footer,.main-content{padding-left:1.5rem;padding-right:1.5rem}.main-wrapper{align-items:center;display:flex;flex:1;flex-direction:column}.font-red{color:#b20f03}.spacer{margin:2rem 0}.h1{font-size:2.5rem;font-weight:500;line-height:3.75rem}.h2{font-weight:500}.core-msg,.h2{font-size:1.5rem;line-height:2.25rem}.body-text,.core-msg{font-weight:400}.body-text{font-size:1rem;line-height:1.25rem}@media (width <= 720px){.h1{font-size:1.5rem;line-height:1.75rem}.h2{font-size:1.25rem}.core-msg,.h2{line-height:1.5rem}.core-msg{font-size:1rem}}#challenge-error-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSI+PHBhdGggZmlsbD0iI2ZjNTc0YSIgZD0iTTE2IDNhMTMgMTMgMCAxIDAgMTMgMTNBMTMuMDE1IDEzLjAxNSAwIDAgMCAxNiAzbTAgMjRhMTEgMTEgMCAxIDEgMTEtMTEgMTEuMDEgMTEuMDEgMCAwIDEtMTEgMTEiLz48cGF0aCBmaWxsPSIjZmM1NzRhIiBkPSJNMTcuMDM4IDE4LjYxNUgxNC44N0wxNC41NjMgOS41aDIuNzgzem0tMS4wODQgMS40MjdxLjY2IDAgMS4wNTcuMzg4LjQwNy4zODkuNDA3Ljk5NCAwIC41OTYtLjQwNy45ODQtLjM5Ny4zOS0xLjA1Ny4zODktLjY1IDAtMS4wNTYtLjM4OS0uMzk4LS4zODktLjM5OC0uOTg0IDAtLjU5Ny4zOTgtLjk4NS40MDYtLjM5NyAxLjA1Ni0uMzk3Ii8+PC9zdmc+);padding-left:34px}#challenge-error-text,#challenge-success-text{background-repeat:no-repeat;background-size:contain}#challenge-success-text{background-image:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIzMiIgaGVpZ2h0PSIzMiIgZmlsbD0ibm9uZSIgdmlld0JveD0iMCAwIDI2IDI2Ij48cGF0aCBmaWxsPSIjMzEzMTMxIiBkPSJNMTMgMGExMyAxMyAwIDEgMCAwIDI2IDEzIDEzIDAgMCAwIDAtMjZtMCAyNGExMSAxMSAwIDEgMSAwLTIyIDExIDExIDAgMCAxIDAgMjIiLz48cGF0aCBmaWxsPSIjMzEzMTMxIiBkPSJtMTAuOTU1IDE2LjA1NS0zLjk1LTQuMTI1LTEuNDQ1IDEuMzg1IDUuMzcgNS42MSA5LjQ5NS05LjYtMS40Mi0xLjQwNXoiLz48L3N2Zz4=);padding-left:42px}.text-center{text-align:center}.pow-button{background-color:#0051c3;border:.063rem solid #0051c3;border-radius:.313rem;color:#fff;font-size:.875rem;line-height:1.313rem;margin:2rem 0;padding:.375rem 1rem;transition-duration:.2s;transition-property:background-color,border-color,color;transition-timing-function:ease}.pow-button:hover{background-color:#003681;border-color:#003681;color:#fff;cursor:pointer}.footer{font-size:.75rem;line-height:1.125rem;margin:0 auto;max-width:60rem;width:100%}.footer-inner{border-top:1px solid #d9d9d9;padding-bottom:1rem;padding-top:1rem}.clearfix:after{clear:both;content:"";display:table}.clearfix .column{float:left;padding-right:1.5rem;width:50%}.diagnostic-wrapper{margin-bottom:.5rem}.footer .ray-id{text-align:center}.footer .ray-id code{font-family:monaco,courier,monospace}.core-msg,.zone-name-title{overflow-wrap:break-word}@media (width <= 720px){.diagnostic-wrapper{display:flex;flex-wrap:wrap;justify-content:center}.clearfix:after{clear:none;content:none;display:initial;text-align:center}.column{padding-bottom:2rem}.clearfix .column{float:none;padding:0;width:auto;word-break:keep-all}.zone-name-title{margin-bottom:1rem}}.loading-spinner{height:76.391px}.lds-ring{display:inline-block;position:relative}.lds-ring,.lds-ring div{height:1.875rem;width:1.875rem}.lds-ring div{animation:lds-ring 1.2s cubic-bezier(.5,0,.5,1) infinite;border:.3rem solid transparent;border-radius:50%;border-top-color:#313131;box-sizing:border-box;display:block;position:absolute}.lds-ring div:first-child{animation-delay:-.45s}.lds-ring div:nth-child(2){animation-delay:-.3s}.lds-ring div:nth-child(3){animation-delay:-.15s}@keyframes lds-ring{0%{transform:rotate(0)}to{transform:rotate(1turn)}}@media screen and (-ms-high-contrast:active),screen and (-ms-high-contrast:none){.main-wrapper,body{display:block}}.rtl .heading-favicon{margin-left:.5rem;margin-right:0}.rtl #challenge-success-text{background-position:100%;padding-left:0;padding-right:42px}.rtl #challenge-error-text{background-position:100%;padding-left:0;padding-right:34px}</style><meta http-equiv="refresh" content="390"></head><body class="no-js"><div class="main-wrapper" role="main"><div class="main-content"><noscript><div id="challenge-error-title"><div class="h2"><span id="challenge-error-text">Enable JavaScript and cookies to continue</span></div></div></noscript></div></div><script>(function(){window._cf_chl_opt={cvId: '3',cZone: "beacon.schneidercorp.com",cType: 'managed',cNounce: '28917',cRay: '89565d36ac397d1a',cHash: '47375b3742c7e0b',cUPMDTk: "\/Application.aspx?AppID=165&LayerID=2145&PageTypeID=2&PageID=1104&KeyValue=0527427230&html.parser&__cf_chl_tk=Of4B4c7stWB5SXvzpcRfYtsBNPOViNaNK0Ll9b5neUY-1718662168-0.0.1.1-4095",cFPWv: 'g',cTTimeMs: '1000',cMTimeMs: '390000',cTplV: 5,cTplB: 'cf',cK: "visitor-time",fa: "\/Application.aspx?AppID=165&LayerID=2145&PageTypeID=2&PageID=1104&KeyValue=0527427230&html.parser&__cf_chl_f_tk=Of4B4c7stWB5SXvzpcRfYtsBNPOViNaNK0Ll9b5neUY-1718662168-0.0.1.1-4095",md: "rmmSJgeD56LpTOvrhy4DfR8o7RU_J0Mb91rJJIbLBXQ-1718662168-1.1.1.1-d25k6KHPu0f_TpOQSmgKRsOCg5vU130QkwpV5TbrFFcsoGgiTQp6MeQOCB5gFbnZVLkOOl_glh6FwjR9WNfDp_Nxb99nyFYlHPSMJLLLZXVUsvEaraTpJ0VYK5vR8RPw6JK4TMBg16JOTm6n8N78f49tUYNsgQVbO09II98fphgiKkrY6MTEDVrDY2QqDrGFrUVz7VD2Y3SjGeC1SmKdH9Jic4qnchy.dj1_DJiAJTLQju.3pS34xv1lkxSzMeoBOG.dSwRMG5NB0oICddop9iYkoUYkiimjE7lenhKgL2W_ZrrSICqiyaXm_2rdXmYQxIGacIUetCO9o2j7Kpw47PKH3lszsqLuQMoXOcws2zYJXjG..0WmLIgkabofpUlVrMnDxKMhuI9tKBjjwS32QH4gpaRG.a0PvXbL_EjqiZYRHXxE6.NFARlbOEnPc5lDJB7nlgdAnkyHYTKHcMmA2a.yyvd.YHloLQD_FP748XMIFq4GScn1a9YxnyP6TW2_yiU9ijT_.mhoGlyQKWuQaq52SK5H2j99N4_uHzm3uQZ234Tv8q0eQ4tEDnCvIWI0Mz9F8DXJFDv.DRl1K95GmpD4D1elwhOQqMS9imBfGGN17FzFWLdBXnG1ZbLUlRQ1XTOd6261ibwTsmo2ALqoFjMVN__mucN1dWcWGTdQTGYefKWORAyn28g8eo5Vj28qy7M3wYqdLulVphF57PY4QgvUS5Lj7GeHO4fI1xDL7VosTxCQYdLHt.kc62ctn7f7q4NmiaIMh_X6O7SzgefYNg_0HcnRDsu4f1hdpRoPCrSqYViL60OQrH9s9rFi1cs9yAdvT5KSRdEVKn6yZlGiKp25lw8kwZijkifdlrbUXjjMB13pVqAApSornOd3ZaB8LeTcmebdPn3uVZ3zefpFEqB.zreEN60ctDFywXHNRQJdemXKKw5en8vH9Z1htdOvSUR.zP.jGhVCd6pA5WeC45KkwcMQlB_FpoETsS8eZIxS5k9YXVSlQXCoO3m.rgayokIdNOeQ4CZMTgPx_C5HeXxpbT9IHRpTbsymZdb3MMzeHY89zFcNY5o6K0OFQPV7mzEv19BKRT7l6bI9USomkV2WixTOeZU_m.u_hPCAGibsMrLRGpJCxWckGpWEP3.Sgu5UuPGlGHviIxx9HtbLdiXnv.pwyP7.RTUkcBYcUCeVf3v2c17envOLFKEqNRclD.ZK4TPUZ8QxbeTKXfnKY3FCrSbaU9hfj0GV9k2A98NgGbn.NxOdppg4i7VlA251LtZsnuHRsGkcPP4vIs0GJMNmro9TLjgpDYoqvMenfaBCimq.eofRK5T.o2E2SixxXujdmnSlBMgg.3gGMgyzV.v7iXEYOpg5OFyxNFp37hNw3xcHdd1cZJs421FE8bGH1HyqFzkvym_qE.mx_.Ut8tK.OxJIHDfdO19YJqieJ9YExUR_9uSBscFznIbT_0ZEAl4VGvmpCRehACvjGuhC4q7.o3aWq1MvL3I_r7JG8YyniNQYfZ_mGNgTu5LGMI0Xnt7Pan.zBK6fXh9k.s3CWneSU_51R0p6yiKydsi_B9gw5eJLxsPBqS5PWDGfIlRV2Hj6v4GyEMezVHtuErfCxbkPoQd5BOFvz0en2SwNfGuAY01hdk2eJWlxUujTHnibT8ph7Oipg8_.AFhlXcrcKkpNHD265ZB9LtBsNaiUumlXf_MIGaKrJFaygM6vf8vVnpzR8Qed4K6ws9WLRME8ENh0gwGfZTZuw26qDoK4M97hLodwatbfBrD3eI_W4Nlp.rIYp1l6GzWLeKWdzA0kAQ",mdrd: "3etA5PVmQCVVM_aEgwlZiVjISxDftXwiCnxf_X7iNsU-1718662168-1.1.1.1-WsAyjo7VoGj0jqSHgoQg0Q9w5sxrSyBSjIIOK68mjmh2mU_tRXXlBf4QodpYxI1XBbA2eMzOWuH0ak4Lefjix59qg0xrz8Ue8czMo86uiGUG6PM04EzUdW3N60Mug9TPFH31V9xAa4xjYSPk8XyYz_NPch3ACGXpoaJrHqNdcGpjdr5Q_f75jrqWIL7JuKza0KapZpWaOQ_yylgCMRVfEvXWmYWkcFPg8Tru4xr6Oarx.yx3fJ1yD1vVoI01k8YG_l8BhAisKNdAPSGN9wxNKka3HKCpaMIwDGhSoO1JOrtyoZFZ0ZoN41RG0GhbojD9TfqNlLcJceq2GEwAum48cuESWLfbqmTvwRJ1_QsQsl2Sz8mHMlKuHqhXjf0garO2bWEMf_gOol8xOxQF_REmhjrKLhWp0UvgZntInAvMulzDsQ2z0LmlxApDvDDEcrIsAS1Gb_e9ochz3tvJVLAyyaMrjqb22.xOwPekNXCcLRcbYmv4Rwp35oUgSLaoRXfuTvZRXCextUuIL8t4YlQpN.KZ3KNjdXOyQRg91VIrjs5DPYSjlcn1DKtUoybSthNLNslrPmGqAGuTcWRCWXAs3dNPlQf2slrzYh9zVoCDO_wYACbt8m5m8q2UUxbDRF2fZetH9JaQCYD3ac6F0MM4MEQHK4NKfZQyJEbZCTP5yTU2Tnl6mPfLbR0vx5JqSQLzEyQvnKb3xFKAf7QBrgSDKhwTssYJ9_pmEbYojDJpjz0xP1kx1PsQY409f6nIMaZHVYFsTl6ivW_6wHuJtL64Ikwdt7JiHRkJnPKjlEwJnToGvHi9MjDP1y_rBOUOQD4DsZzdwkza20ls4i9wZvNGfM8oVqhkMplteNarLSIO6VM5LvaaPJ729twPPT.PV9SEUCCcFFwXba.NZKX5i6gbCytqOuxrL_AzOospag5Oj4RPN3PVI_vRMqYGG7ZSfTbzW9v38WYrz2_seYLyH4.kY8anvNVguqFTBnpi_RwcdwJjkPUMgmnikDtaudhsZpq7RO8ru5TSpgTuonsKZV.ALgmYTHOHTrfzID_2zoQ5SUgoQDkcPt8hmRh8V.yhckcXSgIoLyX4n_MS_7k3Ua.5sosqjL5XKWItOGw7L34lOc3cKubpoFh3Ye.MdVq87yUceA6Z9oyXYSjhrAhBzk5vsrobARiAX2Zc9qlyt8ZNrOF63kcnZhQCEU1xmr1fGwSm17S3eok0fGHr5bVxCQhDUXOXEjsh3lcF_LLKSveWvjI8IHvzTW8eXAuR3aNxatBlvMpRwC9B25.xTn.D2AUj0nV8yeN2GS38FP2ejd1A7g_j2RcrWtrUFp1j7eiuK_wDyyjDE.hSClKZbDA4uTmdw1Rxf1Q6gF_mVH8LxG9azBm5yqAHWke5DEK8qoYfkjOgVwXEL_qwtwu7eyVqD_ZoinrOKDDJwPwJ2KS.bZVfVWAbPXutM8XoI41juZUgJB1xuaI2D.CIls0EdnELw5O0b_BEIiWifzGpohrMWquq8WG0aoDdR8hTet3cHyU5GU535AmCBAZO0VI03sh4YbxJvVBXDPl9EfEZaCe.QFtpc56ej4if7DwOAKvH2dzCKY9fHKOxolu0H5ZOBSnYqcVeAtOfF9fP3INh12jlN338zbAaXsgihoWiALgSvmLo41IxmjUC435vEnFnLhdPvtP10iLWgZeBcaEdTscqd6F5ojTSNhxdwsYrJp3hM2BDpRseSUY_rUAlBWiV4zwERwsQphlnbgfN9fHc6RGdcK.JHW.Ji.y4kATTHIEQKSOxe.Ih0dSu.l8QD54Lv6dnfl8i.W_At0.9G6P4GKPDpgoNRiRvDMh89r5U5kEkNUK7Ly8HqeTJL4hNRZDji13ZN2SUR71XdNWzjyKtMbsHoZacVgqZRLZfZJPF6yY_hIECo99pf7f0Md_oYblwUjcnq4NcstjFm_sDmHmFd69uRODRvyamP8oGI3uK6fOyhuP6_pEfurdEikUz3apG1DO3dx9JKTV0fIcrDbi7gqHuAiNVs22qExP4EE68GrZQykD3RQz64a6gpkR2fIW4m4dJL2R_L9xnHq59iRVYRFh2Py7rwLzvTJIasm0RKtgHpscW8SWViFFTPIxfElDEe_Egc_S61i3gxt1GZf1uPT7s40t3cAw",cRq: {ru: 'aHR0cHM6Ly9iZWFjb24uc2NobmVpZGVyY29ycC5jb20vQXBwbGljYXRpb24uYXNweD9BcHBJRD0xNjUmTGF5ZXJJRD0yMTQ1JlBhZ2VUeXBlSUQ9MiZQYWdlSUQ9MTEwNCZLZXlWYWx1ZT0wNTI3NDI3MjMwJmh0bWwucGFyc2Vy',ra: 'cHl0aG9uLXJlcXVlc3RzLzIuMzEuMA==',rm: 'R0VU',d: 'u4f3Yobgwf5u6zv7Vyez37fDJMRVaKuOq4Y5paFsnFojnKglovP/CR4xXE9H/B8aGKiWmpGN1XHcP0Vz7qLFRqmmeeJP/GfPv83oDcqCgaDYYPTkl8T8DCoO5yplG3tvPrUquNAWpqzfVWiWIZOybBY9qdP5Noun/yrL94UndnQoz8xRNNujnm8jjICAgAVNC6YW4tAAMTbgyLk3y1J1aaCh029b2aCcPaEbI5000bwD2yX6pG8JSfkDfBRWoBJM7O9+WMUzelXC2Xim15NCEyUmQAsRduvTyqn9aVV0vOTJ4KsQHeCpD5+BDuE49N08SoRSpWXoapg8TSEmP6gIQyN8iqw1uX6eoWvVK9eFXuERXmpzBNANK5R3ba3dN+cEsmdkTaAIPjMlewlmn8HqdZV2dqT7mLG+dXp0CcFsRCa3D0l1saSNF5eE57Xsbn/V52VBim3pJZXT/KoWtfE1wllrKE13g12B19oTZnV1aTNn1CzzOpy3510FgxQOl/BpjXIDr07u4uyzmoQJiFNqpE8kqv6Dun5xNhqPDFu+NzXY2P6c6neXnFpR1nsz/67CBJIJaCf7C0isThSlIYB2B3d6H6X2eHXmXpFHEUEghPU=',t: 'MTcxODY2MjE2OC4wMDAwMDA=',cT: Math.floor(Date.now() / 1000),m: 'wDCh8QRn8G04Q366J+IK+KpGMLciSRvuB2gMZ5fhyf0=',i1: '9WrJz2g1uAB/7FMTNOMfKw==',i2: 'JDZfN4djPTXHy+E32CWj5Q==',zh: '9PbOsqhSKsLpjRc2OkMWk0h/2bQLoa45/u0P1gLq0HY=',uh: 'YE9XOpG5TeHmhA1zfs5mxC8CrRZzq2a/+r+OU7dliYQ=',hh: 'z1PLu2KPJfY2I4YzfUIJyk5iSeOJDT8VV7UBO5WLEL4=',}};var cpo = document.createElement('script');cpo.src = '/cdn-cgi/challenge-platform/h/g/orchestrate/chl_page/v1?ray=89565d36ac397d1a';window._cf_chl_opt.cOgUHash = location.hash === '' && location.href.indexOf('#') !== -1 ? '#' : location.hash;window._cf_chl_opt.cOgUQuery = location.search === '' && location.href.slice(0, location.href.length - window._cf_chl_opt.cOgUHash.length).indexOf('?') !== -1 ? '?' : location.search;if (window.history && window.history.replaceState) {var ogU = location.pathname + window._cf_chl_opt.cOgUQuery + window._cf_chl_opt.cOgUHash;history.replaceState(null, null, "\/Application.aspx?AppID=165&LayerID=2145&PageTypeID=2&PageID=1104&KeyValue=0527427230&html.parser&__cf_chl_rt_tk=Of4B4c7stWB5SXvzpcRfYtsBNPOViNaNK0Ll9b5neUY-1718662168-0.0.1.1-4095" + window._cf_chl_opt.cOgUHash);cpo.onload = function() {history.replaceState(null, null, ogU);}}document.getElementsByTagName('head')[0].appendChild(cpo);}());</script></body></html>'

which is very different from:
the result from:
view-source:https://beacon.schneidercorp.com/Application.aspx?AppID=165&LayerID=2145&PageTypeID=4&PageID=1108&KeyValue=0527427230

I’m new to web scraping…

Tags:

2

Answers


  1. 0

    The difference is that when you visit the URL with your browser it processes the scripts that are received from it.

    The call to requests.get() doesn’t do any additional processing thus you see the raw response from the server.

    You can try to use libraries that will parse the returned code to provide you with the final webpage like you see in your browser but it might be simpler to use one of the testing tools like Selenium. With it you can query the page simulating any browser you want, adding potentially important flags and then continue with parsing the result. Here is a simple example:

    from selenium import webdriver

URL = ""

driver = webdriver.Firefox()
driver.get(URL)

with open("webpage.html", 'wb') as f:
    f.write(driver.page_source)

driver.close()
    Login or Signup to reply.
  2. 0

    It seems like the website you are trying to web-scrape, is using cloudflare protection from bots to get into its pages. Also, the website it self built using asp.net that many of its features require javascript.

    So, while request library in python can’t bypass cloudflare protection nor running javascript, then you can’t use it to web-scrape that page.

    Instead, you can use Cloudscraper library that built above request library to bypass cloudflare protection.
    (Visit this to learn more about cloudscraper)

    Another way to do it, is using selenium python library to scrape that pages. Selenium is library that uses your chrome or other chromium-based browsers to visit and interact with webpages. It can work in background or in front as an original window. You can learn about using selenium to web-scrape webpages here.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search