Robot.txt?

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Arthur Pratz

    Robot.txt?

    Hi, Does anyone know about robot.txt?
    Do I really need one for my site?
    If so, how do I get it and what does it do for me?
    Thanks for your time,
    Mike Pratz
    Crafting Classic Mints & Gum with New York Flavor Since 1934 Discover Our ProductsOUR PRODUCTS Bold Flavors. Iconic Look. Made for Moments Worth Savoring. Mints Shop NowGums Shop NowMerchandise Shop NowABOUT US The Candy That Never Followed the Rules C. Howard Company Inc. was started by Charles Howard, of New York City in the early 1930’s. Mr. Howard, in search ...



  • Stan Brown

    #2
    Re: Robot.txt?

    In article <9duCb.444644$0 v4.20770093@bgt nsc04-
    news.ops.worldn et.att.net> in comp.infosystem s.www.authoring.html,
    Arthur Pratz <APratz@worldne t.att.net> wrote:[color=blue]
    >Hi, Does anyone know about robot.txt?[/color]

    That's "robots.txt ", which might explain why you didn't have much
    luck when you tried searching for it on your own.

    Two helpful resources:



    --
    Stan Brown, Oak Road Systems, Cortland County, New York, USA

    HTML 4.01 spec: http://www.w3.org/TR/html401/
    validator: http://validator.w3.org/
    CSS 2 spec: http://www.w3.org/TR/REC-CSS2/
    2.1 changes: http://www.w3.org/TR/CSS21/changes.html
    validator: http://jigsaw.w3.org/css-validator/

    Comment

    • Find Yourself

      #3
      Re: Robot.txt?

      "Arthur Pratz" <APratz@worldne t.att.net> wrote in message
      news:9duCb.4446 44$0v4.20770093 @bgtnsc04-news.ops.worldn et.att.net...[color=blue]
      > Hi, Does anyone know about robot.txt?
      > Do I really need one for my site?
      > If so, how do I get it and what does it do for me?
      > Thanks for your time,
      > Mike Pratz
      > http://www.chowardcompany.com[/color]

      Your UI is wrong on so many levels. At least move the navigation to the
      top.


      Comment

      • Beauregard T. Shagnasty

        #4
        Re: Robot.txt?

        Quoth the raven named Find Yourself:
        [color=blue]
        > "Arthur Pratz" <APratz@worldne t.att.net> wrote in message
        > news:9duCb.4446 44$0v4.20770093 @bgtnsc04-news.ops.worldn et.att.net...
        >[color=green]
        >>Hi, Does anyone know about robot.txt?
        >>Do I really need one for my site?
        >>If so, how do I get it and what does it do for me?
        >>Thanks for your time,
        >>Mike Pratz
        >>http://www.chowardcompany.com[/color]
        >
        >
        > Your UI is wrong on so many levels. At least move the navigation to the
        > top.[/color]

        What navigation?

        <http://www.delorie.com/web/lynxview.cgi?ur l=http%3A%2F%2F www.chowardcomp any.com%2F>

        --
        -bts
        -This space intentionally left blank.

        Comment

        • Tim

          #5
          Re: Robot.txt?

          "Arthur Pratz" <APratz@worldne t.att.net> wrote
          [color=blue][color=green]
          >> http://www.chowardcompany.com[/color][/color]


          "Find Yourself" <null@null.co m> wrote:
          [color=blue]
          > Your UI is wrong on so many levels. At least move the navigation to the
          > top.[/color]

          Why? Isn't it more sensible to read a page before following any links?
          If the links were first, you'd have to scroll back to them.

          Putting links at the top is also bad news for search engine results, as
          a general rule. Pages found by the engine often end being presented as
          just a page title and list of meaningless links, without any useful
          descriptive snippet from the page.

          --
          My "from" address is totally fake. (Hint: If I wanted e-mails from
          complete strangers, I'd have put a real one, there.) Reply to usenet
          postings in the same place as you read the message you're replying to.

          Comment

          • Arthur Pratz

            #6
            Re: Robot.txt?

            Can you explain "UI is wrong on so many levels" in greater detail to me?
            I used Frontpage 02 to create code.
            Thanks for your time.
            Mike Pratz
            Crafting Classic Mints & Gum with New York Flavor Since 1934 Discover Our ProductsOUR PRODUCTS Bold Flavors. Iconic Look. Made for Moments Worth Savoring. Mints Shop NowGums Shop NowMerchandise Shop NowABOUT US The Candy That Never Followed the Rules C. Howard Company Inc. was started by Charles Howard, of New York City in the early 1930’s. Mr. Howard, in search ...





            "Find Yourself" <null@null.co m> wrote in message
            news:bre043$dss $1@news-int2.gatech.edu ...[color=blue]
            > "Arthur Pratz" <APratz@worldne t.att.net> wrote in message
            > news:9duCb.4446 44$0v4.20770093 @bgtnsc04-news.ops.worldn et.att.net...[color=green]
            > > Hi, Does anyone know about robot.txt?
            > > Do I really need one for my site?
            > > If so, how do I get it and what does it do for me?
            > > Thanks for your time,
            > > Mike Pratz
            > > http://www.chowardcompany.com[/color]
            >
            > Your UI is wrong on so many levels. At least move the navigation to the
            > top.
            >
            >[/color]


            Comment

            • Jon Bell

              #7
              Re: Robot.txt?

              In article <qugltvclno3o4r 2bubd7moj57o787 tf364@4ax.com>,
              Tim <dont.byte.me@o ptusnet.com.au> wrote:[color=blue]
              >"Arthur Pratz" <APratz@worldne t.att.net> wrote[color=green]
              >>
              >> Your UI is wrong on so many levels. At least move the navigation to the
              >> top.[/color]
              >
              >Why? Isn't it more sensible to read a page before following any links?
              >If the links were first, you'd have to scroll back to them.[/color]

              What if a visitor comes directly to the page via a search engine, and sees
              quickly (by glancing at the first screenful of information) that it's not
              quite what he wants, but what he wants is likely to be somewhere else on
              the site?

              I put my navigation links both at the top and bottom of the page, for that
              reason. I use plain text links to make a simple breadcrumb trail, so they
              have useful keywords in them, which helps the search engines.

              --
              Jon Bell <jtbellap8@pres by.edu> Presbyterian College
              Dept. of Physics and Computer Science Clinton, South Carolina USA

              Comment

              • Stan Brown

                #8
                Re: Robot.txt?

                In article <YeDCb.447298$0 v4.20820961@bgt nsc04-
                news.ops.worldn et.att.net> in comp.infosystem s.www.authoring.html,
                Arthur Pratz <APratz@worldne t.att.net> wrote:[color=blue]
                >Can you explain "UI is wrong on so many levels" in greater detail to me?
                >I used Frontpage 02 to create code.
                >Thanks for your time.
                >Mike Pratz
                >http://www.chowardcompany.com[/color]

                Before I give my answer to your question, let me point out:

                makes it harder to follow discussions.
                before the material you're commenting on, it
                When you put your comments


                Now, I don't know what the other poster was thinking, but here's
                what I suggest for your site:

                There are (apparently) decorative images of chewing gum, with no
                indication of what they're for or what the site is about, until (on
                my screen anyway) one scrolls down. "CHoward" is not exactly a
                household name. As a general rule you wants _some_ useful content to
                appear "above the fold", i.e. near the top of the page.

                The images don't have alt=" " attributes. What is a blind person, or
                someone using a text browser, supposed to do with your site?

                As far as I can see there are no links to anything in the site,
                except a mailto: for the Webmaster. That last is good, and I wish
                more sites had it; but having no other links is not good. If your
                site is nothing but a home page now, admit that and tell people when
                it will be worth coming back -- then stick to that self-imposed
                deadline. Don't tell people "check back often" -- that's a dead
                giveaway that getting the site done is a low priority for you.

                Lose the "under construction"; it looks amateurish. Every web site
                is under construction, even those that haven't been updated in five
                years.

                --
                Stan Brown, Oak Road Systems, Cortland County, New York, USA

                HTML 4.01 spec: http://www.w3.org/TR/html401/
                validator: http://validator.w3.org/
                CSS 2 spec: http://www.w3.org/TR/REC-CSS2/
                2.1 changes: http://www.w3.org/TR/CSS21/changes.html
                validator: http://jigsaw.w3.org/css-validator/

                Comment

                • Find Yourself

                  #9
                  Re: Robot.txt?

                  "Arthur Pratz" <APratz@worldne t.att.net> wrote in message
                  news:YeDCb.4472 98$0v4.20820961 @bgtnsc04-news.ops.worldn et.att.net...[color=blue]
                  > Can you explain "UI is wrong on so many levels" in greater detail to me?
                  > I used Frontpage 02 to create code.
                  > Thanks for your time.
                  > Mike Pratz
                  > http://www.chowardcompany.com[/color]

                  Ok...
                  First, I apologize for the criticism without compliment. I actually thought
                  the mint/gum packages were kind of eye catching, but when I first visited
                  the page it took me a while to figure out exactly what you do. The only
                  reason I said put some navigation at the top is because in my quest for
                  figuring out your markets and products, I had to figure out where your
                  navigation was. Now I am not sure if this is bad UI or not, but I am going
                  to say it is since the purpose of a website is to inform visitors about the
                  product and other information the company wishes to convey. And as someone
                  mentioned, I guess the links aren't showing up in lynx and other text-based
                  browsers. Now the proportion of visitors that would be using this is
                  probably very minimal, but I would recommend placing some kind of menu
                  somewhere that didn't involve the heavy scripting. Just my $.02.


                  Comment

                  • Alan J. Flavell

                    #10
                    Re: Robot.txt?

                    On Sat, 13 Dec 2003, Jon Bell wrote:
                    [color=blue]
                    > What if a visitor comes directly to the page via a search engine, and sees
                    > quickly (by glancing at the first screenful of information) that it's not
                    > quite what he wants, but what he wants is likely to be somewhere else on
                    > the site?
                    >
                    > I put my navigation links both at the top and bottom of the page, for that
                    > reason.[/color]

                    By your own argument, it might be better to compose an executive
                    summary of the page contents, followed by navigation links (preceded
                    by one which would skip to the main content), followed by the main
                    content itself. And then the navigation links repeated at the foot,
                    as you say.

                    My own choice, however, is to supply the link URLs in <link rel="..."
                    for the more discerning browsers, and the same link URLs at the foot
                    of the page. I don't really care for cluttering up the start of the
                    page itself with navigation (and then having to add that extra link in
                    order to achieve adequate accessibility).

                    OTOH there's something to be said for putting link hrefs at the end
                    and using CSS to propose that they be displayed near the top right of
                    the page. By that means, they don't obstruct accessibility to the
                    content, but they're in a fairly convenient place for the mainstream
                    browsing situation.

                    I never really -did- understand where the widely-observed convention
                    of putting links in a *left*-hand margin came from. It seems to me
                    to be quite a poor choice, out of all the possibilities.

                    (None of these comments apply directly to a page whose chief purpose
                    is to be a menu of links, of course.)
                    [color=blue]
                    > I use plain text links to make a simple breadcrumb trail, so they
                    > have useful keywords in them, which helps the search engines.[/color]

                    Good idea. I never did get around to that yet, I admit.

                    Comment

                    • Beauregard T. Shagnasty

                      #11
                      Re: Robot.txt?

                      Quoth the raven named Find Yourself:[color=blue]
                      > "Arthur Pratz" <APratz@worldne t.att.net> wrote in message
                      > news:YeDCb.4472 98$0v4.20820961 @bgtnsc04-news.ops.worldn et.att.net...
                      >[color=green]
                      >>Can you explain "UI is wrong on so many levels" in greater detail to me?
                      >>I used Frontpage 02 to create code.
                      >>Thanks for your time.
                      >>Mike Pratz
                      >>http://www.chowardcompany.com[/color]
                      >
                      > Ok...[/color]
                      <snippage>
                      [color=blue]
                      > And as someone
                      > mentioned, I guess the links aren't showing up in lynx and other text-based
                      > browsers. Now the proportion of visitors that would be using this is
                      > probably very minimal, but I would recommend placing some kind of menu
                      > somewhere that didn't involve the heavy scripting. Just my $.02.[/color]

                      Not only text browsers: include the ~15% or so that have JavaScript
                      turned off. With js off, there is not only no navigation, there is
                      also no company logo at the top of the page. Just a large area filled
                      with background image, then finally some gum pictures.

                      Prominent company name and purpose, and navigation, is too important
                      to rely on JavaScript.

                      Remember: Google can't find the navigation either.

                      --
                      -bts
                      -This space intentionally left blank.

                      Comment

                      • Stan Brown

                        #12
                        Re: Robot.txt?

                        In article <brfd9o$4kf$1@n ews-int.gatech.edu> in
                        comp.infosystem s.www.authoring.html, Find Yourself wrote:[color=blue][color=green]
                        >> http://www.chowardcompany.com[/color][/color]
                        [color=blue]
                        >And as someone
                        >mentioned, I guess the links aren't showing up in lynx and other text-based
                        >browsers.[/color]

                        "The links"? WHAT links, pray tell? I'm using Mozilla 1.4 -- hardly
                        a "text-based browser" -- and the only link I could find on the
                        entire page was the mailto for the webmaster.

                        --
                        Stan Brown, Oak Road Systems, Cortland County, New York, USA

                        HTML 4.01 spec: http://www.w3.org/TR/html401/
                        validator: http://validator.w3.org/
                        CSS 2 spec: http://www.w3.org/TR/REC-CSS2/
                        2.1 changes: http://www.w3.org/TR/CSS21/changes.html
                        validator: http://jigsaw.w3.org/css-validator/

                        Comment

                        • Arthur Pratz

                          #13
                          Re: Robot.txt?

                          Hi All,
                          I want to thank everyone who has posted.
                          I am learning slowly.
                          The site does have a JS Pulldown Menu bar on the bottom of the page.
                          The top of the page also has the DHTML effects.
                          I am planning on getting rid of the DHTML effects.
                          However, a plain text menu bar would be so many lines of links I didnt like
                          it.
                          If you go to a site like MICROSFT.COM, they have JS menus similar to mine.
                          Again, thanks for your comments.
                          Mike Pratz
                          Crafting Classic Mints & Gum with New York Flavor Since 1934 Discover Our ProductsOUR PRODUCTS Bold Flavors. Iconic Look. Made for Moments Worth Savoring. Mints Shop NowGums Shop NowMerchandise Shop NowABOUT US The Candy That Never Followed the Rules C. Howard Company Inc. was started by Charles Howard, of New York City in the early 1930’s. Mr. Howard, in search ...




                          "Stan Brown" <the_stan_brown @fastmail.fm> wrote in message
                          news:MPG.1a454c 958beeb82698b95 5@news.odyssey. net...[color=blue]
                          > In article <brfd9o$4kf$1@n ews-int.gatech.edu> in
                          > comp.infosystem s.www.authoring.html, Find Yourself wrote:[color=green][color=darkred]
                          > >> http://www.chowardcompany.com[/color][/color]
                          >[color=green]
                          > >And as someone
                          > >mentioned, I guess the links aren't showing up in lynx and other[/color][/color]
                          text-based[color=blue][color=green]
                          > >browsers.[/color]
                          >
                          > "The links"? WHAT links, pray tell? I'm using Mozilla 1.4 -- hardly
                          > a "text-based browser" -- and the only link I could find on the
                          > entire page was the mailto for the webmaster.
                          >
                          > --
                          > Stan Brown, Oak Road Systems, Cortland County, New York, USA
                          > http://OakRoadSystems.com/
                          > HTML 4.01 spec: http://www.w3.org/TR/html401/
                          > validator: http://validator.w3.org/
                          > CSS 2 spec: http://www.w3.org/TR/REC-CSS2/
                          > 2.1 changes: http://www.w3.org/TR/CSS21/changes.html
                          > validator: http://jigsaw.w3.org/css-validator/[/color]


                          Comment

                          • William Tasso

                            #14
                            Re: Robot.txt?

                            Arthur Pratz wrote:[color=blue]
                            > ...
                            > If you go to a site like MICROSFT.COM, they have JS menus similar to
                            > mine.[/color]

                            They don't work either.

                            --
                            William Tasso - http://WilliamTasso.com


                            Comment

                            • Beauregard T. Shagnasty

                              #15
                              Re: Robot.txt?

                              Quoth the raven named Arthur Pratz:
                              [color=blue]
                              > If you go to a site like MICROSFT.COM, they have JS menus similar to mine.[/color]

                              I doubt if anyone here ever said Microsoft did everything right. <g>

                              Consider: http://www.delorie.com/web/lynxview.html

                              If your whole site works in this viewer, e.g. it is navigable and all
                              the content is presented, you're on your way to a good site.

                              --
                              -bts
                              -This space intentionally left blank.

                              Comment

                              Working...