Author Topic: PHP Sitemap - Scan problem  (Read 28857 times)

txmodxoops

  • Newbie
  • *
  • Posts: 30
PHP Sitemap - Scan problem
« on: June 02, 2016, 12:07:51 PM »
Hi

I have this problem!

If when scanning the url ends with / index.php instead, it is going to create the same line another url with two single quotes, another different url with including index.php

Example:
http://www.mysite.org/modules/mymodule/'http://www.mysite.org/index.php'

Thanks for your wonderful work :)

Elmar

  • Administrator
  • Hero Member
  • *****
  • Posts: 2491
  • a command shell is enough to do amazing things
Re: PHP Sitemap - Scan problem
« Reply #1 on: June 02, 2016, 12:10:51 PM »
Hello,

did you try the new version?

Regards Elmar

txmodxoops

  • Newbie
  • *
  • Posts: 30
Re: PHP Sitemap - Scan problem
« Reply #2 on: June 02, 2016, 13:52:55 PM »
Hello,

Yes, it's the same result of the previous versions.

Elmar

  • Administrator
  • Hero Member
  • *****
  • Posts: 2491
  • a command shell is enough to do amazing things
Re: PHP Sitemap - Scan problem
« Reply #3 on: June 02, 2016, 14:25:16 PM »
Please send me the URL of your Website. If you want as private message. I will check it in the next days.

txmodxoops

  • Newbie
  • *
  • Posts: 30
Re: PHP Sitemap - Scan problem
« Reply #4 on: June 02, 2016, 14:26:32 PM »

txmodxoops

  • Newbie
  • *
  • Posts: 30
Re: PHP Sitemap - Scan problem
« Reply #5 on: June 02, 2016, 14:41:32 PM »
I want to alert you that define the left have the quotes (Windows OS)

As php manual: http://php.net/manual/en/function.define.php

Elmar

  • Administrator
  • Hero Member
  • *****
  • Posts: 2491
  • a command shell is enough to do amazing things
Re: PHP Sitemap - Scan problem
« Reply #6 on: June 02, 2016, 14:58:24 PM »
Thanks for the quote info.

txmodxoops

  • Newbie
  • *
  • Posts: 30
Re: PHP Sitemap - Scan problem
« Reply #7 on: June 02, 2016, 17:54:06 PM »
In my error-log, there are many warning

defines that are bottom it's advisable to place them at the top
« Last Edit: June 02, 2016, 17:58:51 PM by txmodxoops »

Elmar

  • Administrator
  • Hero Member
  • *****
  • Posts: 2491
  • a command shell is enough to do amazing things
Re: PHP Sitemap - Scan problem
« Reply #8 on: June 06, 2016, 10:03:52 AM »
I tried the latest Sitemap version 2.0-test1 with your page and it worked without problems.

txmodxoops

  • Newbie
  • *
  • Posts: 30
Re: PHP Sitemap - Scan problem
« Reply #9 on: June 06, 2016, 17:49:37 PM »
I receive these Notice:

Notice: Undefined index: scheme in the line /sitemap_2.0-test1.php file 163
Notice: Undefined index: in /sitemap_2.0-test1.php host file to the line 168
Notice: Undefined index: scheme in the line /sitemap_2.0-test1.php file 163
Notice: Undefined index: in /sitemap_2.0-test1.php host file to the line 168
Notice: Undefined index: scheme in the line /sitemap_2.0-test1.php file 163
Notice: Undefined index: in /sitemap_2.0-test1.php host file to the line 168
Notice: Undefined index: scheme in the line /sitemap_2.0-test1.php file 163
Notice: Undefined index: in /sitemap_2.0-test1.php host file to the line 168
Notice: Undefined index: scheme in the line /sitemap_2.0-test1.php file 163
Notice: Undefined index: in /sitemap_2.0-test1.php host file to the line 168
Notice: Undefined index: scheme in the line /sitemap_2.0-test1.php file 163
Notice: Undefined index: in /sitemap_2.0-test1.php host file to the line 168
Notice: Undefined index: scheme in the line /sitemap_2.0-test1.php file 163
Notice: Undefined index: in /sitemap_2.0-test1.php host file to the line 168
Notice: Undefined index: scheme in the line /sitemap_2.0-test1.php file 163
Notice: Undefined index: in /sitemap_2.0-test1.php host file to the line 168
Notice: Undefined index: scheme in the line /sitemap_2.0-test1.php file 163
Notice: Undefined index: in /sitemap_2.0-test1.php host file to the line 168
Notice: Undefined index: scheme in the line /sitemap_2.0-test1.php file 163
Notice: Undefined index: in /sitemap_2.0-test1.php host file to the line 168
Notice: Undefined index: scheme in the line /sitemap_2.0-test1.php file 163
Notice: Undefined index: in /sitemap_2.0-test1.php host file to the line 168
Notice: Undefined index: scheme in the line /sitemap_2.0-test1.php file 163
Notice: Undefined index: in /sitemap_2.0-test1.php host file to the line 168
Notice: Undefined index: scheme in the line /sitemap_2.0-test1.php file 163
Notice: Undefined index: in /sitemap_2.0-test1.php host file to the line 168
Notice: Undefined index: scheme in the line /sitemap_2.0-test1.php file 163
Notice: Undefined index: in /sitemap_2.0-test1.php host file to the line 168
Notice: Undefined index: scheme in the line /sitemap_2.0-test1.php file 163
Notice: Undefined index: in /sitemap_2.0-test1.php host file to the line 168
Notice: Undefined index: scheme in the line /sitemap_2.0-test1.php file 163
Notice: Undefined index: in /sitemap_2.0-test1.php host file to the line 168
Notice: Undefined index: scheme in the line /sitemap_2.0-test1.php file 163
Notice: Undefined index: in /sitemap_2.0-test1.php host file to the line 168

txmodxoops

  • Newbie
  • *
  • Posts: 30
Re: PHP Sitemap - Scan problem
« Reply #10 on: June 06, 2016, 18:06:42 PM »
I tried the latest Sitemap version 2.0-test1 with your page and it worked without problems.

Compared to the previous version does not compile all the links properly, there are many more

Where do I send the completed files for showing them to you?

Thanks...!

Elmar

  • Administrator
  • Hero Member
  • *****
  • Posts: 2491
  • a command shell is enough to do amazing things
Re: PHP Sitemap - Scan problem
« Reply #11 on: June 06, 2016, 19:38:25 PM »
What PHP version do you use?

Elmar

  • Administrator
  • Hero Member
  • *****
  • Posts: 2491
  • a command shell is enough to do amazing things
Re: PHP Sitemap - Scan problem
« Reply #12 on: June 06, 2016, 20:25:53 PM »
Send me a private message with the file attached.

The notice messages can be ignored. They will be removed in 2.0-test2.

txmodxoops

  • Newbie
  • *
  • Posts: 30
Re: PHP Sitemap - Scan problem
« Reply #13 on: June 07, 2016, 01:07:15 AM »
Php version 5.4

This attached file is with version 2.0 and work fine, but don't create all links

Why this?

txmodxoops

  • Newbie
  • *
  • Posts: 30
Re: PHP Sitemap - Scan problem
« Reply #14 on: June 07, 2016, 01:08:34 AM »
This file is with version 1.0 and you can see the problem!

Don't have an account on GitHub?
« Last Edit: June 07, 2016, 01:14:40 AM by txmodxoops »

Elmar

  • Administrator
  • Hero Member
  • *****
  • Posts: 2491
  • a command shell is enough to do amazing things
Re: PHP Sitemap - Scan problem
« Reply #15 on: June 07, 2016, 06:32:11 AM »
Forget the 1.0 version. Only use 2.0 (test).

Give me some examples what link on what page is missing.

txmodxoops

  • Newbie
  • *
  • Posts: 30
Re: PHP Sitemap - Scan problem
« Reply #16 on: June 25, 2016, 10:59:35 AM »
I send a file to show you my result of version 1.0, the version 2.0 work but not for all links!

I would like to enter your code in xoops module turning it into a class xoops ...!

You let me?

Thank you
« Last Edit: June 25, 2016, 11:31:11 AM by txmodxoops »

Elmar

  • Administrator
  • Hero Member
  • *****
  • Posts: 2491
  • a command shell is enough to do amazing things
Re: PHP Sitemap - Scan problem
« Reply #17 on: June 25, 2016, 13:56:28 PM »
A bug fixed version is available for download.

Best regards
Elmar

txmodxoops

  • Newbie
  • *
  • Posts: 30
Re: PHP Sitemap - Scan problem
« Reply #18 on: June 25, 2016, 19:15:06 PM »
Thanks Elmar  :)

txmodxoops

  • Newbie
  • *
  • Posts: 30
Re: PHP Sitemap - Scan problem
« Reply #19 on: June 26, 2016, 00:51:37 AM »
Results of the new version 2.0 test 2

There is a little bug &

Thanks again!

Elmar

  • Administrator
  • Hero Member
  • *****
  • Posts: 2491
  • a command shell is enough to do amazing things
Re: PHP Sitemap - Scan problem
« Reply #20 on: June 26, 2016, 10:01:25 AM »
Hello,

no, thats not a bug. Its correct encoded for XML.

The '&' is a reserved char in XML and has to be encoded to &
The link on your page is for example  ...../mylinks/singlelink.php?cid=1&lid=12
The '&' has to be encoded and the result for XML is ...../mylinks/singlelink.php?cid=1&lid=12
When the sitemap crawler reads the XML file, then it decodes the '&' back to '&'  and the URL is the same as on your page.


Btw, I think that the ...../mylinks/singlelink.php?cid=1&lid=12 URLs are generated by a module of your site. The browser will translate it to ...../mylinks/singlelink.php?cid=1&lid=12, so its not a problem.


View with your browser the source code of the index page of your site and search for   singlelink.php?cid=1&lid=12   and you will find it.


Regards
Elmar

txmodxoops

  • Newbie
  • *
  • Posts: 30
Re: PHP Sitemap - Scan problem
« Reply #21 on: June 26, 2016, 15:08:38 PM »
Yes, I know what you mean,

but my request was to create a code that eliminates any duplicates of & or amp;

I have this:
Code: [Select]
$link = 'page.php?get=' . urlencode($valore);
echo '<a href="' . htmlentities($link) . '">Here</a>'; 
,

do you think is appropriate to write it in the xoops code modules, or you can put it in your code?

Regards
« Last Edit: June 26, 2016, 15:18:10 PM by txmodxoops »

Elmar

  • Administrator
  • Hero Member
  • *****
  • Posts: 2491
  • a command shell is enough to do amazing things
Re: PHP Sitemap - Scan problem
« Reply #22 on: June 26, 2016, 15:12:35 PM »
My script takes the links as they are in your website code. You have to clean your website code when you don't want the duplicates.

txmodxoops

  • Newbie
  • *
  • Posts: 30
Re: PHP Sitemap - Scan problem
« Reply #23 on: June 26, 2016, 15:32:46 PM »
I have not yet figured out why is only the last url-articles, and not from first to last of the articles ...

it is a problem of access to all the links?

Elmar

  • Administrator
  • Hero Member
  • *****
  • Posts: 2491
  • a command shell is enough to do amazing things
Re: PHP Sitemap - Scan problem
« Reply #24 on: June 26, 2016, 15:43:17 PM »
do you think is appropriate to write it in the xoops code modules, or you can put it in your code?

The clean way is to fix it in your code.


You will end with the &amp; in your html code as long as you use htmlentities for the URL that has an &. I dont know why you use htmlentities for the anchor.
« Last Edit: June 26, 2016, 15:45:00 PM by Elmar »

Elmar

  • Administrator
  • Hero Member
  • *****
  • Posts: 2491
  • a command shell is enough to do amazing things
Re: PHP Sitemap - Scan problem
« Reply #25 on: June 26, 2016, 15:46:21 PM »
You should use htmlentities text output and so on, but not for an anchor.
« Last Edit: June 26, 2016, 15:48:39 PM by Elmar »

txmodxoops

  • Newbie
  • *
  • Posts: 30
Re: PHP Sitemap - Scan problem
« Reply #26 on: June 26, 2016, 15:48:32 PM »
I agree why I have not solved the problem.