No indexing url including query string with Robots txt - Moz

Disallow: the url or folder or other url with conditions you need to block. As you have asked in your question you need to block a url with a condition. But you ... Skiptocontent Mozlogo Menuopen Menuclose Search Products MozPro MozProHome MozLocal MozLocalHome STAT MozscapeAPI FreeSEOTools CompetitiveResearch LinkExplorer KeywordExplorer DomainAnalysis MozBar MoreFreeSEOTools LearnSEO Beginner'sGuidetoSEO SEOLearningCenter MozAcademy SEOQ&A Webinars,Whitepapers,&Guides Blog WhyMoz AgencySolutions EnterpriseSolutions SmallBusinessSolutions CaseStudies TheMozStory NewReleases Login Logout Products MozPro YourAll-In-OneSuiteofSEOTools TheessentialSEOtoolset:keywordresearch,linkbuilding,siteaudits,pageoptimization,ranktracking,reporting,andmore. Learnmore TryMozProfree MozLocal CompleteLocalSEOManagement RaiseyourlocalSEOvisibilitywitheasydirectorydistribution,reviewmanagement,listingupdates,andmore. Learnmore Checkmypresence STAT EnterpriseRankTracking SERPtrackingandanalyticsforSEOexperts,STAThelpsyoustaycompetitiveandagilewithfreshinsights. Learnmore Bookademo MozscapeAPI ThePowerofMozDataviaAPI PoweryourSEOwiththeproven,mostaccuratelinkmetricsintheindustry,poweredbyourindexoftrillionsoflinks. Learnmore Getconnected CompareSEOProducts FreeSEOTools CompetitiveResearch CompetitiveIntelligencetoFuelYourSEOStrategy GainintelonyourtopSERPcompetitors,keywordgaps,andcontentopportunities. Findcompetitors LinkExplorer PowerfulBacklinkDataforSEO Exploreourindexofover40trillionlinkstofindbacklinks,anchortext,DomainAuthority,spamscore,andmore. Getlinkdata KeywordExplorer TheOneKeywordResearchToolforSEOSuccess Discoverthebesttraffic-drivingkeywordsforyoursitefromourindexofover500millionrealkeywords. Searchkeywords DomainAnalysis FreeDomainSEOAnalysisTool GettopcompetitiveSEOmetricslikeDomainAuthority,toppages,rankingkeywords,andmore. Analyzedomain MozBar Free,InstantSEOMetricsAsYouSurf UsingGoogleChrome,seetopSEOmetricsinstantlyforanywebsiteorsearchresultasyoubrowsetheweb. TryMozBar MoreFreeSEOTools LearnSEO Beginner'sGuidetoSEO The#1mostpopularintroductiontoSEO,trustedbymillions. ReadtheBeginner'sGuide How-ToGuides Step-by-stepguidestosearchsuccessfromtheauthorityonSEO. SeeAllSEOGuides SEOLearningCenter BroadenyourknowledgewithSEOresourcesforallskilllevels. VisittheLearningCenter MozAcademy Upskillandgetcertifiedwithon-demandcourses&certifications. ExploretheCatalog On-DemandWebinars LearnmodernSEObestpracticesfromindustryexperts. ViewAllWebinars SEOQ&A Insights&discussionsfromanSEOcommunityof500,000+. FindSEOAnswers August7-9,2023 LockinSuperEarlyBirdsavingsforMozCon Snagtickets Blog WhyMoz SmallBusinessSolutions Uncoverinsightstomakesmartermarketingdecisionsinlesstime. GrowYourBusiness TheMozStory Mozwasthefirst&remainsthemosttrustedSEOcompany. ReadOurStory AgencySolutions Earn&keepvaluableclientswithunparalleleddata&insights. DriveClientSuccess CaseStudies ExplorehowMozdrivesROIwithaproventrackrecordofsuccess. SeeWhat'sPossible EnterpriseSolutions Gainacompetitiveedgeintheever-changingworldofsearch. ScaleYourSEO NewReleases GetthescooponthelatestandgreatestfromMoz. SeeWhat’sNew NewFeature:MozPro Surfaceactionablecompetitiveintel LearnMore Login MozPro MozLocal MozLocalDashboard MozscapeAPI MozscapeAPIDashboard MozAcademy Avatar MozHome Notifications Account&Billing ManageUsers CommunityProfile MyQ&A MyVideos LogOut TheMozQ&AForum Forum Questions Users AsktheCommunity WelcometotheQ&AForum BrowsetheforumforhelpfulinsightsandfreshdiscussionsaboutallthingsSEO. Home SEOTactics TechnicalSEO NoindexingurlincludingquerystringwithRobotstxt NoindexingurlincludingquerystringwithRobotstxt TechnicalSEO 4 7 20348 LoadingMorePosts OldesttoNewest NewesttoOldest MostVotes Reply Replyasquestion Logintoreply Thistopichasbeendeleted.Onlyuserswithquestionmanagementprivilegescanseeit. HMK-NL lasteditedby Dearall, howcanIblockurl/pageswithquerystringslikepage.html?dir=asc&order=namewithrobotstxt? Thanks! 1Reply Lastreply Reply Quote 0 HMK-NL lasteditedby Dearall,whatisthebestoption?Andaretheoptionbelowgood?A:Disallow sort-order(OnlyURLswithvalue=asc) "AsingleURLmaycontainmanyparametersforeachofwhichyoucanspecifysettings.Morerestrictivesettingsoverridelessrestrictivesettings.Forexample,herearethreeparametersandtheirsettings" source: http://support.google.com/webmasters/bin/answer.py?hl=en&answer=1235687 B: User-agent: Googlebot Disallow:/*.=name$ forexample www.sub.domain.com/collection.html?dir=desc&order=namesource:http://support.google.com/webmasters/bin/answer.py?hl=en&answer=156449 Thanks! 1Reply Lastreply Reply Quote 0 kyleNeedham lasteditedby Youcouldalwaysjustuse rel="canonical"whichwouldbemuchbetterthancompletelyblockingallURLparameters. 1Reply Lastreply Reply Quote 0 Matthew_Edgar @HMK-NL lasteditedby Hey, ShouldthatsecondURLbe www.sub.domain.com/collection/adresboeken.html?whatever=somethingIfso,thenbyusing/collection/?*youaresayingthatanythingwithin/collection/withaquerystringshouldnotbeindexed.If adresboeken.htmlalwayshasaquerystring,itmaynotgetindexed. TheotheroptionsI'dconsiderbeforeusingrobots.txtaretellingGoogletoignore dir=desc&order=colorinGoogleWebmasterToolsparameterhandling.Thisisthebestwaytohandlequerystringissues.(AssumingyouaretryingtoinfluenceGoogle.ClearlyGoogleWebmasterToolswon'taffectBing!) AnotherideaistosetacanonicalURLon/collection/adresboeken.htmlreferencing /collection/adresboeken.htmlwithoutthequerystring.ThistellsthesearchenginesthatthequerystringsdonotmakeauniqueURL.(adresboeken.html?dir=desc&order=coloristhesameas adresboeken.html?dir=desc&order=priceisthesameas adresboeken.html?dir=asc&order=coloristhesameas adresboeken.html,andsoon). Ihopethathelps.Thanks, Matthew 1Reply Lastreply Reply Quote 0 cprasad lasteditedby Hi, Robots.txtworksmainlyon2rules.ThoseareUser-agent:andDisallow: User-agent:thenameoftherobotyouneedtoblock Disallow:theurlorfolderorotherurlwithconditionsyouneedtoblock. Asyouhaveaskedinyourquestionyouneedtoblockaurlwithacondition.ButyouhavetorememberthatRobot.txtisgivingsocriticalresultsifyoudidnotuseitcorrectly. Anywayinyourquestion,youwantedto blockurl/pageswithquerystringslikepage.html?dir=asc&order=name soyouhavetousefollowing: User-agent:* Disallow:/*? Sotheabovewillblockalltheurlswithaquestionmark(?)forallthesearchrobots.Thiswillnotblockonlypage.html?dir=asc&order=nameitwillalosblockcomments.html?dir=asc&order=name Souseitsocarefully. Hopethisisthewhatyouhavelookedfor.Ifneedmorehelpyoumayask. Regards Prasad 1Reply Lastreply Reply Quote 0 HMK-NL lasteditedby Dearall, thanksforresponding.IfIhaveapageslike 1.www.sub.domain.com/collection.htmlexists,Iwanttoindexit,and 2.www.sub.domain.com/collection.html?dir=desc&order=colorwhichIdon'twanttoindex Isthisthewaytodothisinderobots.txt?: Disallow:/collection/?* Thanks! 1Reply Lastreply Reply Quote 0 Matthew_Edgar lasteditedby Hi, Hereisanarticleexplaininghowtodothisinrobots.txt: http://sanzon.wordpress.com/2008/04/29/advanced-usage-of-robotstxt-w-querystrings/ Dependingonwhatyouaretryingtodo,itmightalsobeworthinvestigatingparameterhandlinginGoogleWebmasterTools: http://support.google.com/webmasters/bin/answer.py?hl=en&answer=1235687 Thanks, Matthew 1Reply Lastreply Reply Quote 1 1/1 Firstpost Lastpost Gotomynextpost GotaburningSEOquestion? SubscribetoMozProtogainfullaccesstoQ&A,answerquestions,andaskyourown. Startmyfreetrial HaveaQuestion? BrowseQuestions View AllQuestions New(NoResponses) Discussion Answered ProductSupport Unanswered From AllTime Last30Days Last7Days Last24Hours Sortedby LatestQuestions RecentActivity MostThumbsUp MostResponses FewestResponses OldestQuestions Withcategory AllCategories AffiliateMarketing AlgorithmUpdates API Branding Community CompetitiveResearch ContentDevelopment ConversionRateOptimization DigitalMarketing FeatureRequests GettingStarted Image&VideoOptimization IndustryEvents IndustryNews Intermediate&AdvancedSEO InternationalSEO JobsandOpportunities KeywordExplorer KeywordResearch LinkBuilding LinkExplorer LocalListings LocalSEO LocalWebsiteOptimization MozBar MozLocal MozNews MozPro MozTools On-PageOptimization OtherSEOTools PaidSearchMarketing ProductSupport Reporting&Analytics Research&Trends ReviewsandRatings SearchBehavior SEOTactics SERPTrends SocialMedia TechnicalSEO WebDesign WhiteHat/BlackHatSEO Exploremorecategories MozTools ChatwiththecommunityabouttheMoztools. SEOTactics DiscusstheSEOprocesswithfellowmarketers Community Discussindustryevents,jobs,andnews! DigitalMarketing ChatabouttacticsoutsideofSEO Research&Trends Diveintoresearchandtrendsinthesearchindustry. Support Connectonproductsupportandfeaturerequests. Seeallcategories RelatedQuestions Redirect_toinURLs? I'veneverseenthisbefore.I'massumingthatit'snotSEOfriendlyandthattheseshouldbe301sor302sinstead? http://ksa-beta.motory.com/ar/login/?redirect_to=http://ksa-beta.motory.com/ar/cars-for-sale-search/results/central/riyadh/ford/explorer/2010/ford-explorer-2010-1038353 http://ksa-beta.motory.com/ar/login/?redirect_to=http://ksa-beta.motory.com/ar/account/my-saved-searches/ TechnicalSEO| |KatherineWatierOng 0 GoogleisnotindexingmynewURLstructure.Whynot? Hiall, WelaunchedanewwebsiteforacustomeronApril29th.Thatsamedayweresubmittedthenewsitemap&askedGoogletofetchthenewwebsite. Screenshotisattachedofthis(GWTIndexed). However,whenIlookatGoogleIndex(seeattachment-GoogleIndex),AutomatedProduction'soldwebsiteURL'sstillappear.It'sbeentwoweeks. IsitnormalforGoogle'sindextotakethislongtoupdate? Thanksforyourhelp. Cole VoLPjhyvfxVUsO TechnicalSEO| |ColeLusby 0 Parkedformercompany'surlontopofmyexistingurlandthatURLisshowinginSERPsformytopkeywords IhavetheURLfrommyformercompanyparkedontopofmyexistingURL. MytopkeywordsareshowingupwiththeoldURLattachedtothemetadsecriptionofmyexistingURL. Itwassupposedtobe301redirectedinsteadofparkedbutmywebdeveloperinsiststhiswastherightwaytodoitanditwillworkitselfoutaftergoogleindexestheoldURLoutofexistence. Arethereanyotheroptions? TechnicalSEO| |Joelabarre 0 Robots.txt www.mywebsite.com**/details/**home-to-mome-4596 www.mywebsite.com**/details/**home-moving-4599 www.mywebsite.com**/details/**1-bedroom-apartment-4601 www.mywebsite.com**/details/**4-bedroom-apartment-4612 Wehavesomanypageslikethis,wedonotwanttoGooglecrawlthispages Sowe addedthefollowingcodetoRobots.txt User-agent:Googlebot Disallow:/details/ Thiscodeiscorrect? TechnicalSEO| |iskq 0 Bestwaytohandleindexedpagesyoudon'twantindexed We'vehadalotofpagesindexedbygooglewhichwedidn'twantindexed.Theyrelatetoaajaxcategoryfiltermodulethatworksokforfrontendcustomersbutunderthebonnetgooglehasbeenfollowingallofthelinks. I'veputaruleintherobots.txtfiletostopgooglefromfollowinganydynamicpages(witha?)andalsoanyajaxpagesbutthepagesarestillindexedongoogle. Atthemomentthereisover5000pageswhichhavebeenindexedwhichIdon'twantonthereandI'mworriediscausingissueswithmyrankings. Wouldaredirectruleworkorcouldsomeoneofferanyadvice? https://www.google.co.uk/search?q=site:outdoormegastore.co.uk+inurl:default&num=100&hl=en&safe=off&prmd=imvnsl&filter=0&biw=1600&bih=809#hl=en&safe=off&sclient=psy-ab&q=site:outdoormegastore.co.uk+inurl%3Aajax&oq=site:outdoormegastore.co.uk+inurl%3Aajax&gs_l=serp.3...194108.194626.0.194891.4.4.0.0.0.0.100.305.3j1.4.0.les%3B..0.0...1c.1.SDhuslImrLY&pbx=1&bav=on.2,or.r_gc.r_pw.r_qf.&fp=ff301ef4d48490c5&biw=1920&bih=860 TechnicalSEO| |gavinhoman 0 Bingindexing Hello,people~ IwanttodiscussaboutBingindexation. Ihaveanewwebsitewhichopenedabout3monthsago. GooglehasnoproblemtoindexmysiteandallpageswithinthesiteindexedbyGoogle. However,BingandYahooisdifferentstory. Iusedmanualsubmission,BingwebmastertooltoletBingknowaboutthesite. However,Bingisnotindexingmysiteyet. IresearchedaboutitandfoundthatmysiteshouldhavesomeexternallinksbeforeIgetindexbyBing. IcheckexternallinksofmysitewithGooglewebmastertool,SEOmoztooland"link:"onGoogle. Alltoolsshowdifferentnumberasbelow. GooglewebmasterTool:morethan50 SEMozsiteexplorer:5 link:onGoogle:none Whyallmethodofcheckinglinksaredifferentandwhichonshouldmostdependon? AlsohowmanylinksshouldIhaveinordertogetindexbyBing? Couldyoupeoplepleaseshareyouropinion? TechnicalSEO| |Artience 0 Robots.txtforsubdomain HithereMozzers! IhaveasubdomainwithduplicatecontentandI'dliketoremovethesepagesfromthemightyGoogleindex.Theproblemis:thewebsiteisbuildinDrupalandthissubdomaindoesnothaveit'sownrobots.txt. SoIwanttoaskyouhowtodisallowandnoindexthissubdomain.Isitpossibletoaddthistotherootrobots.txt: User-agent:* Disallow:/subdomain.root.nl/ User-agent:Googlebot Noindex: /subdomain.root.nl/ Thankyouinadvance! Partouter TechnicalSEO| |Partouter 0 RelcanonicalwithindexfollowonquerystringURLs Higuys, Quickquestionregardingtherelcanonicaltag. IhavelotsoflinkspointingatmewithquerystringsandpreviouslyusedsomecodetodetermineifquerystringswereintheURLandiftheywerethennottoindexthatpage.Ifthereweren'tquerystringsthenthepagewouldbeindexedandfollowed. IassumeIcannowusetherelcanonicaltagoneachofthesepagessothevaluegoestotheproperURLminusanyquerystring.HoweverdoIneedtohavetherelcanonicaltagabovetheindex,followtagonthepage? SoURLissite.com/page.html?ref=ABC metarobotsis"index,follow" Relcanonicalis"site.com/page.html" Doestheorderofthemetarobotsandcanonicaltagmatter? Thanksinadvance! TechnicalSEO| |panini 0 × LookslikeyourconnectiontoMozwaslost,pleasewaitwhilewetrytoreconnect.